Systems and methods for suggesting meta-information to a camera user

ABSTRACT

According to one embodiment of the invention, a camera captures an image. The image is transmitted to a server for image recognition processing. The camera receives information from the server, including an indication of information to suggest to a user for meta-tagging the image. The suggested information may be based, for example, on a comparison of the image with meta-information stored by the server and/or a database of stored images. The camera asks the user if the user would like to meta-tag the image with the information. Optionally, the camera receives an indication from the user that the user would like to meta-tag the image with the suggested information, and the camera meta-tags the image with the information.

The present Application claims the benefit of U.S. ProvisionalApplication Ser. No. 60/434,475 filed Dec. 18, 2002, in the name ofWalker et al. The entirety of this provisional application isincorporated by reference herein for all purposes.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a block diagram of a system that is consistent with atleast one embodiment of the present invention.

FIG. 2 shows a block diagram of a system that is consistent with atleast one embodiment of the present invention.

FIG. 3 shows a block diagram of a camera in communication with acomputing device that is consistent with at least one embodiment of thepresent invention.

FIG. 4 shows a block diagram of a computing device that is consistentwith at least one embodiment of the present invention.

FIG. 5 shows a block diagram of a camera that is consistent with atleast one embodiment of the present invention.

FIG. 6 shows a block diagram of a camera that is consistent with atleast one embodiment of the present invention.

FIG. 7 is a table illustrating an exemplary data structure of a settingsdatabase consistent with at least one embodiment of the presentinvention.

FIG. 8 is a table illustrating an exemplary data structure of an imagedatabase consistent with at least one embodiment of the presentinvention.

FIG. 9 is a table illustrating an exemplary data structure of a questiondatabase consistent with at least one embodiment of the presentinvention.

FIG. 10 is a table illustrating an exemplary data structure of adetermination condition database consistent with at least one embodimentof the present invention.

FIG. 11 is a table illustrating an exemplary data structure of an outputcondition database consistent with at least one embodiment of thepresent invention.

FIGS. 12A and 12B are a table illustrating an exemplary data structureof a response database consistent with at least one embodiment of thepresent invention.

FIG. 13A is a table illustrating an exemplary data structure of an eventlog corresponding to capturing images at a wedding, in accordance withat least one embodiment of the present invention.

FIG. 13B is a table illustrating an exemplary data structure of an eventlog corresponding to capturing images on a sunny beach, in accordancewith at least one embodiment of the present invention.

FIG. 14 is a table illustrating an exemplary data structure of anexpiring information database consistent with at least one embodiment ofthe present invention.

FIG. 15 is a flowchart illustrating a process consistent with at leastone embodiment of the present invention.

FIG. 16 is a flowchart illustrating a process consistent with at leastone embodiment of the present invention for performing an action basedon a response.

FIG. 17 is a flowchart illustrating a process consistent with at leastone embodiment of the present invention for performing an action basedon a response.

FIG. 18 is a flowchart illustrating a process consistent with at leastone embodiment of the present invention for suggesting meta-information.

FIG. 19 is a flowchart illustrating a process consistent with at leastone embodiment of the present invention.

FIG. 20 is a flowchart illustrating a process consistent with at leastone embodiment of the present invention.

DETAILED DESCRIPTION

Applicants have recognized that, in accordance with some embodiments ofthe present invention, some types of users of cameras and other imagingdevices may find it appealing to have a camera that is able to determinea variety of different types of information that may be useful inperforming a variety of functions and/or assisting a user in theperformance of various actions. Also, some types of users may find itappealing to use a camera having enhanced features to facilitateinformation gathering (e.g., via interaction with a user, by detectionof environmental conditions, by communication with other devices). Inaccordance with some embodiments, such information may be used, forexample, in managing images (e.g., suggesting a meta-tag for an image)and in improving the quality of images (e.g., by adjusting a camerasetting).

Applicants have also recognized that some types of users of cameras andother imaging devices may find it appealing to be able to receive avariety of different types of questions (e.g., open-ended questions)and/or suggestions (e.g., suggested meta-data to associate with animage) from a camera, as provided for in accordance with at least oneembodiment of the present invention. Some types of users may also findit appealing to be able to provide responses to questions output by acamera.

Some users of cameras (e.g., casual users) seldom adjust their camerasto capture images in the best possible way. Also, even automaticcameras, for example, may still make mistakes in estimating what imagesa user wants to capture and what settings are best for capturing images.Further, even if a user knows how to adjust his camera correctly, he mayoccasionally forget to adjust his camera when he is capturing images.Accordingly, Applicants have recognized that some types of users mayfind it appealing to use a camera having an interface that is convenientand not time consuming for a user to adjust his camera and that mayoptionally suggest (or automatically make) settings adjustments, asprovided for in some embodiments of the present invention.

At least one embodiment of the invention includes a camera that mayoutput questions to a user. The user may respond to these questions(e.g., providing information about a scene that he is interested inphotographing) and one or more settings on the camera may be adjustedbased on the user's response.

For example, a camera may ask a user: “Are you at the beach?” If theuser responds “Yes” to this question, then the camera may adjust one ormore of its settings (e.g., aperture, shutter speed, white balance,automatic neutral density) based on the user's response. In a secondexample, the camera may ask a user a plurality of questions, startingwith “Are you indoors?” If the user responds that he is indoors, thenthe camera may ask the user a second question: “What type of lights doesthis room have?” In addition to outputting the question, the camera mayoutput a list of potential answers to the question (e.g., “Fluorescent,”“Tungsten,” “Halogen,” “Skylight,” and “I don't know”). The user mayrespond to the question by selecting one of the potential answers fromthe list. For example, if the user responds “Fluorescent” to thisquestion, then the camera may adjust its settings to “Fluorescent Light”mode, in which the camera's white balance, aperture, shutter speed,image sensor sensitivity and other settings are adjusted for takingpictures in a room that is lit with fluorescent light bulbs.

Various embodiments of the present invention are described herein withreference to the accompanying drawings. The leftmost digit(s) of areference numeral typically identifies the figure in which the referencenumeral first appears.

Embodiments of the present invention will first be introduced by meansof block diagrams of exemplary systems and devices that may be utilizedby an entity practicing the present invention. Exemplary data structuresillustrating tables that may be used when practicing various embodimentsof the present invention will then be described, along withcorresponding flowcharts that illustrate exemplary processes withreference to the exemplary devices, systems, and tables.

1. Systems and Devices

Referring now to FIG. 1, a block diagram of a system 100 according to atleast one embodiment of the present invention includes one or moreservers 110 (e.g., a personal computer, a Web server) in communication,via a communications network 120, with one or more cameras 130 (e.g.,digital camera, video camera, wireless phone with integrated digitalcamera). Each of the servers 110 and cameras 130 may comprise one ormore computing devices, such as those based on the Intel Pentium®processor, that are adapted to communicate with any number and type ofdevices (e.g., other cameras and/or servers) via the communicationsnetwork 120. Although only two cameras 130 and two servers 110 aredepicted in FIG. 1, it will be understood that any number and type ofcameras 130 may communicate with any number of servers 110 and/or othercameras 130 (and vice versa).

According to one or more embodiments of the present invention, a camera130 may communicate with a server 110 in order to determine a questionto output to a user. For example, the camera 130 may transmit variousinformation (e.g., images, GPS coordinates) to a computer server 110.The server 110 may then determine a question based on this information.The server 110 may then transmit the question to the camera 130 and thecamera 130 may output the question to a user.

Communication among the cameras 130 and the servers 110 may be direct ormay be indirect, and may occur via a wired or wireless medium. Some, butnot all, possible communication networks that may comprise network 120(or may otherwise be part of system 100 and/or other exemplary systemsdescribed herein) include: a local area network (LAN), a wide areanetwork (WAN), the Internet, a telephone line, a cable line, a radiochannel, an optical communications line, and a satellite communicationslink. In yet other embodiments, the devices of the system 100 maycommunicate with one another over RF, cable TV, satellite links and thelike. Some possible communications protocols that may be part of system100 include, without limitation: Ethernet (or IEEE 802.3), SAP, ATP,Bluetooth™, IEEE 802.11, CDMA, TDMA, ultra-wideband, universal serialbus (USB), and TCP/IP. Optionally, communication may be encrypted toensure privacy and to prevent fraud in any of a variety of ways wellknown in the art. Of course, in lieu of or in addition to the exemplarycommunications means described herein, any appropriate communicationsmeans or combination of communications means may be employed in thesystem 100 and in other exemplary systems described herein.

For example, communication may take place over the Internet through aWeb site maintained by a server 110 on a remote server, or over anon-line data network including commercial on-line service providers,bulletin board systems and the like. In another example, using thewireless capabilities of his mobile phone, a user may upload an imagecaptured using the integrated digital camera to his personal computer,or to a personal database of images on a Web server maintained by histelecommunications company. In another example, while a user is stillaway from home on vacation, the user's personal computer may receive,via a cable modem, a series of vacation snapshots taken by the user, andmay also transmit information about those snapshots and/or questionsrelated to those snapshots back to the user's digital camera.

According to one or more embodiments of the present invention, a server110 may comprise an external or internal module associated with one ormore of the cameras 130 that is capable of communicating with one ormore of the cameras 130 and of directing the one or more cameras 130 toperform one or more functions. For example, a server 110 may beconfigured to execute a program for controlling one or more functions ofa camera 130 remotely. Similarly, a camera 130 may comprise a moduleassociated with one or more servers 110 that is capable of directing oneor more servers 110 to perform one or more functions. For example, acamera 130 may be configured to direct a server 110 to execute a facialrecognition program on a captured image and to return an indication ofthe best matches to the camera 130 via the communication network 120.

A camera 130 may be operable to access one or more databases (e.g., ofserver 110) to provide suggestions and/or questions to a user of thecamera 130 based on, for example, an image captured by the camera 130 oron information gathered by the camera 130 (e.g., information aboutlighting conditions). A camera 130 may also be operable to access adatabase (e.g., an image database) via the network 120 to determine whatmeta-information (e.g., information descriptive of an image) toassociate with one or more images. For example, as discussed furtherherein, a database of images and/or image templates may be stored for auser on a server 110. Various functions of a camera 130 and/or theserver 110 may be performed based on images stored in a personalizeddatabase. For instance, an image recognition program running on theserver 110 (or on the camera 130) may use the user's personalizeddatabase of images for reference in identifying people, objects, and/orscenes in an image captured by the user. If, in accordance with apreferred embodiment, the user has identified the content of some of theimages in the database himself (e.g., by associating a meta-tag with animage), a match determined by the image recognition software withreference to the customized database is likely to be acceptable to theuser (e.g., the user is likely to agree to a suggestion to associate ameta-tag from a stored reference image with the new image also).

Information exchanged by the exemplary devices depicted in FIG. 1 mayinclude, without limitation, images and indications of changes insettings or operation of a camera 130 (e.g., an indication that a useror the camera 130 has altered an exposure setting). Other exemplarytypes of information that may be determined by the camera 130 and/or theserver 110 and communicated to one or more other devices are describedherein. The server 110, for example, may monitor operations of a camera130 (and/or activity of a user) via the network 120. For instance, theserver 110 may identify a subject a user is recording images of and,optionally, use that information to direct the camera 130 to ask if theuser would like to e-mail or otherwise transmit a copy of the capturedimage to the subject.

With respect to the various exemplary systems, devices, and methodsdiscussed herein, those skilled in the art will understand that devicesin communication with each other need not be continually transmitting toeach other. On the contrary, such devices need only transmit to eachother as necessary, and may actually refrain from exchanging data mostof the time. For example, a device in communication with another devicevia the Internet may not transmit data to the other device for weeks ata time.

According to some embodiments, various processes may be performed by thecamera 130 in conjunction with the server 110. For example, some stepsof a described process may be performed by the camera 130, while othersteps are performed by the server 110. As discussed herein, data usefulin providing some of the described functionality may be stored on one ofor both of the camera 130 and server 110 (or other devices).

In some embodiments, as discussed herein, the servers 110 may not benecessary and/or may not be preferred. For example, some embodiments ofthe present invention may be practiced using a camera 130 alone, asdescribed herein. In such embodiments, one or more functions describedas being performed by the server 110 may be performed by the camera 130,and some or all of the data described as being stored on a server 110may be stored on the camera 130 or on another device in communicationwith the camera 130 (e.g., another camera, a personal digital assistant(PDA)).

Similarly, in some embodiments the cameras 130 may not be necessaryand/or may not be preferred. Accordingly, one or more functionsdescribed herein as being performed by the camera 130 may be performedby the server 110, and some or all of the described as being stored onthe camera 130 may be stored on the server 110 or on another device incommunication with the server 110 (e.g., a PDA, a personal computer).

A server 110 may be embodied in a variety of different forms, including,without limitation, a mainframe computer (e.g., an SGI Origin™ server),a personal computer (e.g., a Dell Dimension™ computer), and a portablecomputer (e.g., an Apple iBook™ laptop, a Palm m515™ PDA, a Kyocera7135™ cell phone). Several examples of types of cameras, servers, andother devices are discussed herein, and other types consistent withvarious embodiments of the present invention will be readily understoodby those of skill in the art in light of the present disclosure.

Referring now to FIG. 2, a block diagram of a system 200 according to atleast one embodiment of the present invention includes an imaging device210 in communication (e.g., via a communications network or system bus)with a computing device 220. Various exemplary means by which devicesmay communicate are discussed above with respect to FIG. 1. Althoughonly one imaging device 210 and one computing device 220 are depicted inFIG. 2, it will be understood that any number and type of imagingdevices 210 may communicate with any number of computing devices 220.

Various types of imaging devices 210 and computing devices 220 arediscussed herein. The imaging device 210 preferably comprises at leastone device or component for recording an image, such as, withoutlimitation, an image sensor, a camera, or a handheld device having anintegrated camera. It will be understood, therefore, that a lens and animage sensor, for example, may each be referred to individually as animaging device, or, alternatively, two or more such components may bereferred to collectively as an imaging device (e.g., as embodied in acamera or PDA). Further, it will be understood, as discussed furtherbelow with respect to FIG. 3, that a device embodying any suchcomponents (e.g., a camera) may itself be referred to as an imagingdevice.

The imaging device 210 may further comprise one or more types ofcomputing devices, such as those based on the Intel Pentium® processor,adapted to communicate with the computing device 220. For example, aswill be readily apparent to those skilled in the art, many types ofcameras include an imaging device (e.g., an image sensor for capturingimages) and a computing device (e.g., a processor for executing camerafunctions). For example, referring now to FIG. 3, a block diagram of asystem 300 according to at least one embodiment of the present inventionincludes a camera 310 in communication (e.g., via a communicationsnetwork) with a server 340. The camera 310 itself comprises an imagingdevice 320 (e.g., an image sensor and/or lens) and a computing device330 (e.g., a camera processor) that is in communication (e.g., via acommunication port of the computing device 330) with the server 340(e.g., a Web server). It will be understood that a device such as thecamera 310, comprising both an imaging device and a computing device,may itself be referred to, alternatively, as an imaging device or acomputing device.

Referring again to FIG. 2, a computer or computing device 220 maycomprise one or more processors adapted to communicate with the imagingdevice 210 (or one or more computing devices of the imaging device 210).As discussed herein, a computer or computing device 220 preferably alsocomprises a memory (e.g., storing a program executable by the processor)and may optionally comprise a communication port (e.g., forcommunication with an imaging device 210). Some examples of a computeror computing device 220 include, without limitation: a camera processor,a camera, a server, a PDA, a personal computer, a computer server,personal computer, portable hard drive, digital picture frame, or otherelectronic device. Thus, a computing device 220 may but need not includeany devices for capturing images. Some exemplary components of acomputing device are discussed in further detail below with respect toFIGS. 4-6.

In some exemplary embodiments of the present invention, as discussedherein, imaging device 210 comprises a camera (e.g., a camera 130 ofFIG. 1) and the computing device 220 comprises a server (e.g., a server110 of FIG. 1). In another example consistent with at least oneembodiment of the present invention, the system 200 depicts componentsof a camera or other device capable of recording images. For instance,the imaging device 210 may comprise an image sensor or lens incommunication via a camera system bus with a computing device 220 suchas a camera computer or integrated communication device (e.g., a mobilephone).

An imaging device 210 or camera 310 may communicate with one or moreother devices (e.g., computing device 220, server 340) in accordancewith one or more systems and methods of the invention. Examples ofdevices that an imaging device may communicate with include, withoutlimitation:

-   -   (i) a personal digital assistant (PDA)    -   (ii) a cellular telephone    -   (iii) a digital wallet (e.g., the iPod™ by Apple, the MindStor™        from Minds@Work, Nixvue's Digital Album™)    -   (iv) a portable stereo (e.g., an MP3 music player, a Sony        Discman™)    -   (v) a notebook computer    -   (vi) a tablet computer    -   (vii) a digital picture frame (e.g., Iomega's FotoShow™,        NORDview's Portable Digital Photo Album™)    -   (viii) a GPS device (e.g., such as those manufactured by Garmin)    -   (ix) a personal computer

According to various embodiments of the present invention, an imagingdevice 210 may transfer one or more images to a second device (e.g.,computing device 220). Some examples are provided with reference toFIGS. 1-3. In one example, an imaging device 210 may include a wirelesscommunication port that allows the camera to transmit images to a secondelectronic device (e.g., a computer server, personal computer, portablehard drive, digital picture frame, or other electronic device). Thesecond electronic device may then store copies of the images. Aftertransferring the images to this second electronic device, the imagingdevice 210 may optionally delete the images, since the images are nowstored securely on the second electronic device.

According to another exemplary embodiment, the camera 310 may include acellular telephone or be connected to a cellular telephone with wirelessmodem capabilities (e.g., a cellular telephone on a 2.5G or 3G wirelessnetwork). Using the cellular telephone, the camera may transmit one ormore images to the computer server 340, which may store the images.

In another example, an imaging device 210 may communicate with aportable hard drive such as an Apple iPod™. To free up memory on theimaging device 210, the imaging device 210 may transfer images to theportable hard drive.

In another example, the camera 130 may have a wireless Internetconnection (e.g., using the 802.11 wireless protocol) and use thisconnection to transmit images to a personal computer that is connectedto the Internet. Note that by transferring an image from a camera to asecond electronic device, the camera may effectively expand itsavailable memory. That is, some or all of the memory on the secondelectronic device may be available to the camera for storing images.

According to various embodiments of the present invention, a camera 310or other imaging device 210 may communicate with an electronic device tooutput a question to a user. For example, a camera may transmit aquestion to a user's PDA. The question may then be displayed to the userby the PDA. Using a PDA or other device with a relatively large displaymay make it easier for a user to view a question (e.g., a question thatincludes a large amount of text or a question which is based on animage).

In another example, a digital camera may queue up a plurality ofquestions and output these questions to a user's personal computer whenthe user uploads photos from the camera to the personal computer. Thepersonal computer may run software that outputs the questions to theuser and enables the user to respond to the questions. Viewing questionson a personal computer may be more convenient than viewing questionsusing the digital camera. Of course, a user's response to a question maybe less useful to the camera (e.g., in adjusting settings on the camera)if this response is provided after the user has already finishedcapturing images.

According to some embodiments, a camera or other imaging device maycommunicate with an electronic device to receive an input to a user. Forexample, a user may use a PDA to indicate a response to a question andthen the PDA may transmit an indication of this response to the camerausing a Bluetooth communication link. For example, a user may highlighta portion of an image, select a response from a list of responses, orwrite a free form response using the stylus on his PDA. Providing aninput to the camera using a PDA or other electronic device may beparticularly convenient for a user because the PDA may include one ormore input devices that are not present on the camera (e.g., a touchscreen, a GPS device).

In another example, a user may carry a GPS device that is separate fromthe camera but that communicates with the camera using a USB cable. Inorder to indicate his location, the user may transmit an indication ofhis latitude and longitude from the GPS device to the camera. In yetanother example, all user control of a camera may be implemented througha user's cellular telephone. For example, the user may use his cellulartelephone to remotely operate the camera, pressing the “1” and “2” keysto zoom in and zoom out, the “3” key to capture a picture, and the “4”and “5” keys to answer “Yes” and “no” to questions output by the camera.One advantage of having a second device implement a large number ofcontrols for the camera is that the camera can have a very small formfactor, but still be operable by a large number of controls because allof these controls are on the second device.

1.1. Computing Device

Referring now to FIG. 4, illustrated therein is a block diagram of anembodiment 400 of computing device 220 (FIG. 2) or computing device 330(FIG. 3). The computing device 400 may be implemented as a systemcontroller, a dedicated hardware circuit, an appropriately programmedgeneral-purpose computer, or any other equivalent electronic, mechanicalor electro-mechanical device. The computing device 400 may comprise, forexample, a server computer operable to communicate with one or moreclient devices, such as an imaging device 210. The computing device 400may be operative to manage the system 100, the system 200, the system300, and/or the camera 310 and to execute various methods of the presentinvention.

In operation, the computing device 400 may function under the control ofa user, remote operator, image storage service provider, or other entitythat may also control use of an imaging device 210 and/or computingdevice 220. For example, the computing device 400 may be a Web servermaintained by an Internet services provider, or may be a computerembodied in a camera 310 or camera 130. In some embodiments, thecomputing device 400 and an imaging device 210 may be different devices.In some embodiments, the computing device 400 and the imaging device 210may be the same device. In some embodiments, the computing device 400may comprise more than one computer operating together.

The computing device 400 comprises a processor 405, such as one or moreIntel Pentium® processors. The processor 405 is in communication with amemory 410 and with a communication port 495 (e.g., for communicatingwith one or more other devices).

The memory 410 may comprise an appropriate combination of magnetic,optical and/or semiconductor memory, and may include, for example,Random Access Memory (RAM), Read-Only Memory (ROM), a compact discand/or a hard disk. The processor 405 and the memory 410 may each be,for example: (i) located entirely within a single computer or otherdevice; or (ii) connected to each other by a remote communicationmedium, such as a serial port cable, telephone line or radio frequencytransceiver. In one embodiment, the computing device 400 may compriseone or more devices that are connected to a remote server computer formaintaining databases.

The memory 410 stores a program 415 for controlling the processor 405.The processor 405 performs instructions of the program 415, and therebyoperates in accordance with the present invention, and particularly inaccordance with the methods described in detail herein. The program 415may be stored in a compressed, uncompiled and/or encrypted format. Theprogram 415 furthermore includes program elements that may be necessary,such as an operating system, a database management system and “devicedrivers” for allowing the processor 405 to interface with computerperipheral devices. Appropriate program elements are known to thoseskilled in the art, and need not be described in detail herein.

The term “computer-readable medium” as used herein refers to any mediumthat participates in providing instructions to processor 405 (or anyother processor of a device described herein) for execution. Such amedium may take many forms, including but not limited to, non-volatilemedia, volatile media, and transmission media. Non-volatile mediainclude, for example, optical or magnetic disks, such as memory 410.Volatile media include dynamic random access memory (DRAM), whichtypically constitutes the main memory. Transmission media includecoaxial cables, copper wire and fiber optics, including the wires thatcomprise a system bus coupled to the processor 405. Transmission mediacan also take the form of acoustic or light waves, such as thosegenerated during radio frequency (RF) and infrared (IR) datacommunications. Common forms of computer-readable media include, forexample, a floppy disk, a flexible disk, hard disk, magnetic tape, anyother magnetic medium, a CD-ROM, DVD, any other optical medium, punchcards, paper tape, any other physical medium with patterns of holes, aRAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip orcartridge, a carrier wave as described hereinafter, or any other mediumfrom which a computer can read.

Various forms of computer-readable media may be involved in carrying oneor more sequences of one or more instructions to processor 405 (or anyother processor of a device described herein) for execution. Forexample, the instructions may initially be borne on a magnetic disk of aremote computer. The remote computer can load the instructions into itsdynamic memory and send the instructions over a telephone line using amodem. A modem local to a computing device 400 (or, e.g., a server 340)can receive the data on the telephone line and use an infraredtransmitter to convert the data to an infrared signal. An infrareddetector can receive the data carried in the infrared signal and placethe data on a system bus for processor 405. The system bus carries thedata to main memory, from which processor 405 retrieves and executes theinstructions. The instructions received by main memory may optionally bestored in memory 510 either before or after execution by processor 405.In addition, instructions may be received via communication port 495 aselectrical, electromagnetic or optical signals, which are exemplaryforms of carrier waves that carry data streams representing varioustypes of information. Thus, the computing device 400 may obtaininstructions in the form of a carrier wave.

According to an embodiment of the present invention, the instructions ofthe program 415 may be read into a main memory from anothercomputer-readable medium, such from a ROM to RAM. Execution of sequencesof the instructions in program 415 causes processor 405 to perform theprocess steps described herein. In alternate embodiments, hard-wiredcircuitry may be used in place of, or in combination with, softwareinstructions for implementation of the processes of the presentinvention. Thus, embodiments of the present invention are not limited toany specific combination of hardware and software.

The memory 410 also preferably stores a plurality of databases,including a settings database 420, an image database 425, a questiondatabase 430, a determination condition database 435, an outputcondition database 440, a response database 445, an event log 450, andan expiring information database 455. Examples of each of thesedatabases is described in detail below and example structures aredepicted with sample entries in the accompanying figures.

As will be understood by those skilled in the art, the schematicillustrations and accompanying descriptions of the sample databasespresented herein are exemplary arrangements for stored representationsof information. Any number of other arrangements may be employed besidesthose suggested by the tables shown. For example, even though eightseparate databases are illustrated, the invention could be practicedeffectively using any number of functionally equivalent databases.Similarly, the illustrated entries of the databases represent exemplaryinformation only; those skilled in the art will understand that thenumber and content of the entries can be different from thoseillustrated herein. Further, despite the depiction of the databases astables, an object-based model could be used to store and manipulate thedata types of the present invention and, likewise, object methods orbehaviors can be used to implement the processes of the presentinvention.

Note that, although these databases are described with respect to FIG. 4as being stored in one computing device, in other embodiments of thepresent invention some or all of these databases may be partially orwholly stored in another device, such as one or more imaging devices210, one or more of the cameras 130, one or more of the servers 110 or340, another device, or any combination thereof. Further, some or all ofthe data described as being stored in the databases may be partially orwholly stored (in addition to or in lieu of being stored in the memory410 of the computing device 400) in a memory of one or more otherdevices.

1.2. Camera

Referring now to FIG. 5, illustrated therein is a block diagram of anembodiment 530 of a camera (e.g., camera 130 of FIG. 1, camera 310 ofFIG. 3) in communication (e.g., via a communications network) with aserver 550. The camera 530 may be implemented as a system controller, adedicated hardware circuit, an appropriately configured computer, or anyother equivalent electronic, mechanical or electro-mechanical device.The camera 530 may comprise, for example, any of various types ofcameras well known in the art, including, without limitation, a stillcamera, a digital camera, an underwater camera, and a video camera. Astill camera, for example, typically includes functionality to captureimages that may be displayed individually. A single lens reflex (SLR)camera is one example of a still camera. A video camera typicallyincludes functionality to capture movies or video (i.e., one or moresequences of images typically displayed in succession). A still image,movie file or video file may or may not include or be associated withrecorded audio. It will be understood by those skilled in the art thatsome types of cameras, such as the Powershot A40™ by Canon U.S.A., Inc.,may include functionality to capture movies and functionality to capturestill images.

The camera 530 may comprise any or all of the cameras 130 of system 100(FIG. 1) or the imaging device 210 (FIG. 2). In some embodiments, a userdevice such as a PDA or cell phone may be used in place of, or inaddition to, some or all of the camera 530 components depicted in FIG.5. Further, a camera may comprise a computing device or other deviceoperable to communicate with another computing device (e.g., a server110).

The camera 530 comprises a processor 505, such as one or more IntelPentium™ processors. The processor 505 is in communication with a memory510 and a communication port 520 (e.g., for communicating with one ormore other devices). The memory 510 may comprise an appropriatecombination of magnetic, optical and/or semiconductor memory, and mayinclude, for example, Random Access Memory (RAM), Read-Only Memory(ROM), a programmable read only memory (PROM), a compact disc and/or ahard disk. The memory 510 may comprise or include any type ofcomputer-readable medium. The processor 505 and the memory 510 may eachbe, for example: (i) located entirely within a single computer or otherdevice; or (ii) connected to each other by a remote communicationmedium, such as a serial port cable, telephone line or radio frequencytransceiver. In one embodiment, the camera 530 may comprise one or moredevices that are connected to a remote server computer for maintainingdatabases.

According to some embodiments, memory 510 of camera 530 may comprise animage buffer (e.g., a high-speed buffer for transferring images from animage sensor) and/or a flash memory (e.g., a high-capacity, removableflash memory card for storing images). A wide variety of different typesof memory are possible and are known to those skilled in the art. Forexample, memory may be volatile or non-volatile; may be electronic,capacitive, inductive, or magnetic in nature; and may be accessedsequentially or randomly.

Memory may or may not be removable from a camera. Many types of camerasmay use one or more forms of removable memory, such as chips, cards,and/or discs, to store and/or to transfer images and other data. Someexamples of removable media include CompactFlash™ cards, SmartMedia™cards, Sony Memory Sticks™, MultiMediaCards™ (MMC) memory cards, SecureDigital™ (SD) memory cards, IBM Microdrives™, CD-R and CD-RW recordablecompact discs, and DataPlay™ optical media.

The memory 510 stores a program 515 for controlling the processor 505.The program 515 may comprise instructions (e.g., Digital® imagingsoftware, image recognition software) for capturing images and/or forone or more other functions. The processor 505 performs instructions ofthe program 515, and thereby operates in accordance with the presentinvention, and particularly in accordance with the methods described indetail herein. The program 515 may be stored in a compressed, uncompiledand/or encrypted format. The program 515 furthermore includes programelements that may be necessary, such as an operating system, a databasemanagement system and “device drivers” for allowing the processor 505 tointerface with computer peripheral devices. Appropriate program elementsare known to those skilled in the art, and need not be described indetail herein.

According to one embodiment of the present invention, the instructionsof the program 515 may be read into a main memory from anothercomputer-readable medium, such from a ROM to RAM. Execution of sequencesof the instructions in program 515 causes processor 505 to perform theprocess steps described herein. In alternate embodiments, hard-wiredcircuitry may be used in place of, or in combination with, softwareinstructions for implementation of the processes of the presentinvention. Thus, embodiments of the present invention are not limited toany specific combination of hardware and software. As discussed withrespect to system 100 of FIG. 1, execution of sequences of theinstructions in a program of a server 110 in communication with camera530 may also cause processor 505 to perform some of the process stepsdescribed herein.

The memory 510 optionally also stores one or more databases, such as theexemplary databases described in FIG. 4. An example of a camera memory510 storing various databases is discussed herein with respect to FIG.6.

The processor 505 is preferably also be in communication with one ormore imaging devices 535 (e.g., a lens, an image sensor) embodied in thecamera 530. Various types of imaging devices are discussed herein and inparticular with respect to FIG. 6.

The processor 505 is preferably also in communication with one or moreinput devices 525 (e.g., a button, a touch screen) and output devices540. Various types of input devices and output devices are describedherein and in particular with respect to FIG. 6.

Such one or more output devices 540 may comprise, for example, an audiospeaker (e.g., for outputting a question to a user), an infra-redtransmitter (e.g., for transmitting a suggested meta-tag to a user'sPDA), a display device (e.g., a liquid crystal display (LCD)), a radiotransmitter, and a printer (e.g., for printing an image).

An input device 525 is capable of receiving an input (e.g., from a useror another device) and may be a component of camera 530. An input devicemay communicate with or be part of another device (e.g. a server, aPDA). For cameras, common input devices include a button or dial. Someother examples of input devices include: a keypad, a button, a handle, akeypad, a touch screen, a microphone, an infrared sensor, a voicerecognition module, a motion detector, a network card, a universalserial bus (USB) port, a GPS receiver, a radio frequency identification(RFID) receiver, an RF receiver, a thermometer, a pressure sensor, andan infra-red port (e.g., for receiving communications from with a secondcamera or another device such as a smart card or PDA of a user).

Referring now to FIG. 6, illustrated therein is a more detailed blockdiagram of a embodiment 600 of a camera (e.g., camera 130 of FIG. 1,camera 530 of FIG. 5). The camera 630 comprises a processor 605, such asone or more Intel Pentium™ processors. The processor 605 is incommunication with a memory 610 and a communication port 695 (e.g., forcommunicating with one or more other devices). The memory 610 maycomprise or include any type of computer-readable medium, and stores aprogram 615 for controlling the processor 605. The processor 605performs instructions of the program 615, and thereby operates inaccordance with various processes of the present invention, andparticularly in accordance with the methods described in detail herein.

The memory 610 stores a plurality of databases, including a settingsdatabase 620, an image database 625, a question database 630, adetermination condition database 635, an output condition database 640,a response database 645, an event log 650, and an expiring informationdatabase 655. Examples of each of these databases is described in detailbelow and example structures are depicted with sample entries in theaccompanying figures.

The processor 605 is preferably also in communication with a lens 660(e.g., made of glass), an image sensor 665, one or more controls 670(e.g., an exposure control), one or more sensors 675, one or more outputdevices 680 (e.g., a liquid crystal display (LCD)), and a power supply685 (e.g., a battery, a fuel cell, a solar cell). Various examples ofthese types of components are described herein.

A processor of a camera 600 may be capable of executing instructions(e.g., stored in memory 610) such as software (e.g., for wireless and/ordigital imaging, such as Digital® software from Flashpoint Technology,Inc.).

As indicated in FIG. 6, a camera may include one or more input devicescapable of receiving data, signals, and indications from varioussources. Lenses, sensors, communication ports and controls are wellknown types of input devices.

Various types of lenses that may be used with cameras are well known,including telephoto, wide-angle, macro, and zoom lenses.

As will be understood by those of skill in the art, an image sensor maybe an area that is responsive to light and may be used to capture animage. An image sensor may or may not be an electronic device. Someexamples of image sensors include, without limitation: a CCD (ChargeCoupled Device) and a CMOS (Complementary Metal Oxide Semiconductor)image sensor, such as the X3® PRO 10M™CMOS image sensor by Foveon. Animage sensor may comprise software or other means for imageidentification/recognition. “Image sensor” may be most often used torefer to an electronic image sensor, but those skilled in the art willrecognize that various other technologies (e.g., a light sensitive filmlike that used in analog cameras) may also function as image sensors.

A camera may include one or more output devices. Examples of outputdevices include, without limitation: a display (e.g., a color orblack-and-white liquid crystal display (LCD) screen), an audio speaker(e.g., for outputting questions), a printer (e.g., for printing images),a light emitting diode (LED) (e.g., for indicating that a self-timer isfunctioning, for indicating that a question for the user is pending),and a touch screen. A display may be useful, for example, for displayingimages and/or for displaying camera settings.

The camera may also include one or more communication ports for use incommunicating with one or more other devices. For example, a USB(universal serial bus) or Firewire® (IEEE-1394 standard) connection portmay be used to exchange images and other types of data with a personalcomputer or digital wallet (e.g., an Apple iPod™). The camera may be incommunication with a cellular telephone, personal digital assistant(PDA) or other wireless communications device. Images and other data maybe transmitted to and from the camera using this wireless communicationsdevice. For example, the SH251I™ cellular telephone by Sharp Corporationincludes a 3.1 megapixel CCD camera, and allows users to receive imagefiles via e-mail. In yet another example, a camera may include a radioantenna for communicating with a radio beacon. For instance, a subjectof a photo may carry a radio beacon that may communicate with the cameraand provide information that is useful in determining settings for thecamera (e.g., information about the light incident on the subject).

As will be understood by those skilled in the art, a camera may includeone or more controls or other input devices. Examples of controlsinclude, without limitation: a button (e.g., a shutter button), a switch(e.g., an on/off switch), a dial (e.g., a mode selection dial), akeypad, a touch screen, a microphone, a bar code reader (e.g., such asthe one on the 1991 version of the Canon EOS Elan™), a remote control(e.g., such as the one on the Canon Powershot G2™), a sensor, atrackball, a joystick, a slider bar, and a continuity sensor.

Controls on a camera or other type of imaging device may be used toperform a variety of functions. In accordance with various embodimentsof the present invention, a control may be used, without limitation, toadjust a setting or other parameter, provide a response to a question,or operate the camera. For example, a user may press the shutter buttonon the camera to capture an image. Controls may be used to adjust one ormore settings on the camera. For example, a user may use “up” and “down”buttons on a camera to adjust the white balance on the camera. Inanother example, a user may use a mode dial on the camera to select aplurality of settings simultaneously. For example, a user may use acontrol to indicate to the camera that he would like a question to beoutput as an audio recording, or to any adjust any of various othertypes of parameters of how the camera is to operate and/or interact withthe user. As discussed herein controls may be used to provide anindication to the camera. For example, a user may use a control toindicate that he would like to have a question output to him. In stillanother example, a user may use a control to provide a response to aquestion or to provide other information, such as indicating that theuser is in a room with fluorescent lights, at a beach, or capturingimages of a football game.

Various types of sensors that may be included in a camera include,without limitation: a light sensor, an image sensor, a range sensor(e.g., for determining the distance to a subject), a microphone (e.g.,for recording audio that corresponds to a scene), a global positioningsystem (GPS) device (e.g., for determining a camera's location), acamera orientation sensor (e.g., an electronic compass, a tilt sensor,an altitude sensor, a humidity sensor, a clock (e.g., indicating thetime of day, day of the week, month, year), and a temperature/infraredsensor.

According to some embodiments, a microphone may be useful for allowing auser to control the camera using voice commands. Voice recognitionsoftware (e.g., ViaVoice™ from IBM Voice Systems) is known to thoseskilled in the art and need not be described further herein.

1.3. Databases

As will be understood by those skilled in the art, a setting for acamera may be a parameter that affects how the camera operates (e.g.,how the camera captures at least one image). Examples of types ofsettings on a camera include, without limitation: exposure settings,lens settings, digitization settings, flash settings, multi-framesettings, power settings, output settings, function settings, and modesettings. Some more detailed examples of these types of settings arediscussed further below.

Exposure settings may affect the exposure of a captured image. Examplesof exposure settings include, without limitation: shutter speed,aperture, image sensor sensitivity (e.g., measured as ISO or ASA), whitebalance, color hue, and color saturation. Lens settings may affectproperties of a lens on the camera. Examples of lens settings include,without limitation: focus (e.g., near or far), optical zoom (e.g.,telephoto, wide angle), optical filters (e.g., ultraviolet, prism), anindication of which lens to use (e.g., for a camera that has multiplelenses) or which portion of a lens, field of view, and imagestabilization (e.g., active or passive image stabilization).

Digitization settings may affect how the camera creates a digitalrepresentation of an image. Examples of digitization settings include,without limitation: resolution (e.g., 1600x1200 or 640x480), compression(e.g., for an image that is stored in JPG format), colordepth/quantization, digital zoom, and cropping. For instance, a croppingsetting may indicate how the camera should crop an acquired digitalimage when storing it to memory.

Flash settings may affect how the flash on the camera operates. Examplesof flash settings include, without limitation: flash brightness, red-eyereduction, and flash direction (e.g., for a bounce flash). Multi-framesettings may affect how the camera captures a plurality of relatedimages. Examples of multi-frame settings include, without limitation: aburst mode (e.g., taking a plurality of pictures in response to onepress of the shutter button), auto-bracketing (e.g., taking a pluralityof pictures with different exposure settings), a movie mode (e.g.,capturing a movie), and image combination (e.g., using Canon'sPhotoStitch™ program to combine a plurality of images into a singleimage).

Power settings may affect the supply of power to one or more of thecamera's electronic components. Examples of power settings include,without limitation: on/off and “Power-Save” mode (e.g., varioussubsystems on a camera may be put into “Power-Save” mode to prolongbattery life).

Output settings may affect how the camera outputs information (e.g., toa user, to a server, to another device). Examples of output settingsinclude, without limitation: language (e.g., what language is used tooutput prompts, questions, or other information to a user), viewfindersettings (e.g., whether a digital viewfinder on the camera is enabled,how a heads-up-display outputs information to a user), audio outputsettings (e.g., whether the camera beeps when it captures an image,whether questions may be output audibly), and display screen settings(e.g., how long the camera displays images on its display screen aftercapturing them).

In accordance with one or more embodiments of the present invention, acamera may be operable to capture images and to perform one or more of avariety of other functions. A function setting may cause one or morefunctions to be performed (and/or prevent one or more functions frombeing performed). For example, if an auto-rotate setting on a camera isenabled, then the camera may automatically rotate a captured image sothat it is stored and displayed right side up, even if the camera washeld at an angle when the image was captured. Other examples offunctions that may be performed by a camera include, without limitation:modifying an image (e.g., cropping, filtering, editing, addingmeta-tags), cropping an image (e.g., horizontal cropping, verticalcropping, aspect ratio), rotating an image (e.g., 90 degrees clockwise),filtering an image with a digital filter (e.g., emboss, remove red-eye,sharpen, add shadow, increase contrast), adding a meta-tag to an image,displaying an image (e.g., on a LCD screen of the camera), andtransmitting an image to another device (e.g., a personal computer, aprinter, a television).

One way to adjust a setting on the camera is to change the camera'smode. For example, if the camera were to be set to “Fluorescent Light”mode, then the settings of the camera would be adjusted to the exemplaryvalues listed in this column (i.e., the aperture would be set toautomatic, the shutter speed would be set to 1/125 sec, the film speedwould be set to 200 ASA, etc.).

In accordance with some embodiments of the present invention, a moderefers to one or more parameters that may affect the operation of thecamera. A setting may be one type of parameter. Indicating a mode to thecamera may be a convenient way of adjusting a plurality of settings onthe camera (e.g., as opposed to adjusting each setting individually).There are many types of modes. Some types, for example, may affectsettings (e.g., how images are captured) and other modes may affectoutputting questions. Some exemplary modes are discussed herein, withoutlimitation, and other types of modes will be apparent to those skilledin the art in light of the present disclosure. A “Sports” mode, forexample, may describe settings appropriate for capturing images ofsporting events (e.g., fast shutter speeds). For instance, a user mayoperate a control (e.g., a dial) to indicate that the camera should bein “Sports” mode, in which the shutter speed on the camera is fasterthan 1/250 sec and burst capturing of three images is enabled. Anexemplary “Fluorescent Light” mode may establish settings appropriatefor capturing images under fluorescent lights (e.g., white balance). A“Sunny Beach” mode may describe settings appropriate for capturingimages on sunny beaches, and a “Sunset” mode may describe settingsappropriate for capturing images of sunsets (e.g., neutral densityfilter). An exemplary “Portrait” mode may establish settings appropriatefor capturing close-up images of people (e.g., adjusting for skintones).

Referring now to FIG. 7, an exemplary tabular representation 700illustrates one embodiment of settings database 420 (or settingsdatabase 620) that may be stored in an imaging device 210 and/orcomputing device 220. The tabular representation 700 of the settingsdatabase includes a number of example records or entries, each defininga setting that may be enabled or adjusted on an imaging device such ascamera 130 or camera 600. Those skilled in the art will understand thatthe settings database may include any number of entries.

The tabular representation 700 also defines fields for each of theentries or records. The exemplary fields specify: (i) a setting 705,(ii) a current value 710 that indicates the present value or state ofthe corresponding setting, (iii) a value 715 that indicates anappropriate value for when the camera is in a “Fluorescent Light” mode,(iv) a value 720 that indicates an appropriate value for when the camerais in a “Sunny Beach” mode, and (v) a value 725 that indicates anappropriate value for when the camera is in a “Sunset” mode.

The settings database may be useful, for example, for determining thecurrent value 710 of a given setting (e.g., “aperture”). Also, asdepicted in tabular representation 700, one or more values may beestablished for association with a given mode. For example, tabularrepresentation 700 indicates that if the mode of the camera is“Fluorescent Light,” the “aperture” setting will be changed to “auto.”

Referring now to FIG. 8, an exemplary tabular representation 800illustrates one embodiment of image database 425 (or image database 625)that may be stored, for example, in a server 110 and/or camera 130. Thetabular representation 800 of the image database includes a number ofexample records or entries, each defining a captured image. Thoseskilled in the art will understand that the image database may includeany number of entries.

The tabular representation 800 also defines fields for each of theentries or records. The exemplary fields specify: (i) an imageidentifier 805 that uniquely identifies an image, (ii) an image format810 that indicates the file format of the image, (iii) an image size815, (iv) a file size 820, (v) a time 825 that indicates when the imagewas captured, and (vi) meta-data 830 that indicates any of various typesof supplemental information (e.g., keyword, category, subject,description, location, camera settings when the image was captured)associated with the image.

It will be understood by those skilled in the art that a variety ofdifferent types of meta-data 830 are possible, including position (e.g.,GPS), orientation, altitude, exposure settings (aperture/shutter speed),illumination (daylight/tungsten/florescent/IR/flash), lens setting(distance/zoom position/macro), scene data (blue sky/water/grass/faces),subject motion, image content (e.g., subjects), sound annotations, dateand time, preferred cropping, and scale. Other types of meta-data arediscussed herein.

With respect to the image identifier 805, a camera may automaticallyassign an identifier to an image, or a user may use a control (e.g., akeypad) on a camera to indicate an identifier for an image.

The image database may be useful for various types of processesdescribed herein.

In accordance with various embodiments of the present invention, acamera and/or server may output various different types of questions toa user.

A question may comprise a request for information from a user. Forexample, a camera may output a question to a user in order to determineinformation useful in applying meta-information to an image or incapturing one or more images (e.g., information about lighting,information about subjects, information about a scene).

Examples of different types of questions include, without limitation:questions about lighting, questions about people and subjects of images,questions about focus and depth of field, questions about meta-taggingand sorting, questions about events and locations, questions about theenvironment, questions about scenes, questions about future plans,questions about priorities, and questions about images.

Some examples of questions about lighting include, without limitation:

-   -   (i) “Are you indoors?”    -   (ii) “What kind of light bulb(s) does this room have?” (e.g.,        with multiple choice answers “tungsten,” “fluorescent,”        “halogen,” and “I don't know”)    -   (iii) “Are we in the shade?”    -   (iv) “It's very dark here. Is it nighttime?”

Some examples of questions about people and subjects of images include:

-   -   (i) “Who is in this picture?”    -   (ii) “What other pictures is <person> in?”    -   (iii) “It looks like you're taking a picture of Alice. Are you        taking a picture of Alice?”    -   (iv) “Is this the same person that was in picture #42<show        picture #42>?”    -   (v) “It looks like you're taking a picture of an animal. Are you        taking a picture of your pet?”

Examples of questions about focus and depth of field include:

-   -   (i) “What are you trying to focus on?”    -   (ii) “The background of this image looks complicated. Do you        want the background to be in focus?”    -   (iii) “Do you want the background to be sharp or blurred?”    -   (iv) “Do you want <object> to be in focus?”    -   (v) “Do you want <object> to be sharp or blurred?”    -   (vi) “There's a small object in the foreground (left side) of        your image. Do you want this object to be in focus?”

Examples of questions about meta-tagging and sorting:

-   -   (i) “Do you want to automatically store images of <subject> in a        separate directory?”    -   (ii) “Where should I store images of <subject>?”    -   (iii) “How would you characterize this image?”

Examples of questions about events and locations include, withoutlimitation:

-   -   (i) “I think I see candles. Are we at a birthday party?”    -   (ii) “Are you taking pictures of a sporting event?”    -   (iii) “It appears that you're taking lots of pictures of        animals. Are you at the zoo?”    -   (iv) “I saw a bright flash and heard an explosion. Are you        taking pictures of fireworks?”

Examples of questions about the environment include, without limitation:

-   -   (i) “The current humidity level of the air is 90%. Is it raining        outside?”    -   (ii) “This image seems cloudy. Am I underwater?”    -   (iii) “The sky has an orange tint. Is the sun setting?”    -   (iv) “Is it cloudy outside?”    -   (v) “Are we in the shade?”    -   (vi) “The camera is rocking back and forth. Are you on a boat?”    -   (vii) “We're traveling at 200 mph. Are you in an airplane?”    -   (viii) “Are you in a moving vehicle (e.g., a car)?    -   (ix) “Is the camera on a tripod?”

Some examples of questions about scenes include:

-   -   (i) “I think I see a rainbow. Are you trying to take a picture        of a rainbow?”    -   (ii) “I think I see running water. Are you trying to take a        picture of a waterfall?”    -   (iii) “I think I see sand. Are you trying to take a picture of a        beach?”    -   (iv) “I think I see a body of water. Are you trying to take a        picture of an ocean, lake, or pond?”    -   (v) “Are you taking a picture through a window?”    -   (vi) “I think I see a reflection. Will this picture have a        mirror in it?    -   (vii) “Are you taking a picture of a sunset?”    -   (viii) “Are you taking a picture of stars in the sky?”    -   (ix) “What is the most important element in this scene?”    -   (x) “Are you taking a picture of a reflection?”

Examples of questions about future plans include:

-   -   (i) “There are 23 Mb of memory remaining. How many more pictures        are you planning on taking today?”    -   (ii) “How many pictures are you planning on taking of this        scene?”    -   (iii) “You've already captured 23 pictures of Alice. How many        pictures are you planning on taking of Alice?”    -   (iv) How much longer are you planning on using the camera?        (e.g., with multiple choice answers “less than 10 minutes,”        “10-30 minutes,” “30-60 minutes,” and “more than 60 minutes”)    -   (v) “This memory card has only 10 Mb of space left. Do you have        any blank memory cards with you?”    -   (vi) “My batteries will run out in less than 30 minutes. Do you        have any more charged batteries with you?”    -   (vii) “Are you done taking pictures of this scene?”    -   (viii) “Do you need a good picture of every subject?”    -   (ix) “Who do you want to capture pictures of today?”    -   (x) “Who should receive a copy of this image? (e.g., Grandma,        Uncle Joey, Kodak.com's Picture of the Day contest)”    -   (xi) “Are you planning on using this image in a slide show?”    -   (xii) “Are you planning on emailing this image to somebody?”

Some examples of questions about priorities include, without limitation:

-   -   (i) “Are you more concerned about exposure or sharpness?”    -   (ii) “Are you more concerned about framing or resolution?”    -   (iii) “I think I see Alice and Bob. Who is the subject of this        photo? (e.g., with multiple choice answers “Alice,” “Bob,” “both        Alice and Bob,” and “neither Alice nor Bob”)”    -   (iv) “Do you want to focus on Alice or the mountains?”

Some examples of questions about images include:

-   -   (i) “Who is the subject of this image?”    -   (ii) “How would you rate this image on a 1-10 scale?”    -   (iii) “How would you rate the exposure of this image on a 1-10        scale?”    -   (iv) “Would you prefer the background of this image to be        sharper or more blurred?”    -   (v) “Is this an image of Alice or Amy?”    -   (vi) “Does Alice's skin tone look correct in this image?”    -   (vii) “Which is your favorite picture from this group?”    -   (viii) “Rank these images from best to worst, starting with your        favorite image.”    -   (ix) “Is this image overexposed?”    -   (x) “Should this image be categorized automatically?”

Different types of questions may illicit different types of responsesfrom a user. For example, questions may be classified according to thetypes of responses they are designed to illicit. Some examples ofquestions classified in this manner include:

-   -   (i) Yes/No questions (e.g., “Is this a picture of Alice?”)    -   (ii) open-ended questions (e.g., “Who is in this picture?”)    -   (iii) multiple-choice questions in which the user is presented        with a plurality of options and prompted to choose at least one        of the options (e.g., “Who is in this picture? a) Alice b)        Bob c) both d) neither.”)    -   (iv) true/false questions (e.g., “True or False?: This is a        picture of Alice.”)    -   (v) graphical response questions (e.g., “Point to Alice in this        picture.”)    -   (vi) ratings (e.g., a user may be asked to rate how much he        likes an image)

Note that a question may be phrased in the first person. Some types ofusers may find this personification of the camera appealing. Variousexemplary ways that a question may be output to a user are discussedherein.

Referring now to FIG. 9, an exemplary tabular representation 900illustrates one embodiment of the question database 430 (FIG. 4) thatmay be stored in the computing device 400. The tabular representation900 of the question database includes a number of example records orentries, each defining a question that may be output by a server 110 orby a camera 130. Those skilled in the art will understand that thequestion database may include any number of entries.

The tabular representation 900 also defines fields for each of theentries or records. The exemplary fields specify: (i) a questionidentifier 905 that uniquely identifies a particular question, (ii) aquestion to ask 910 that includes an indication (e.g., in text) of aquestion to output (e.g., to a user, to transmit to a camera 130), and(iii) a potential response 915 that indicates one or more potentialresponses to the corresponding question (e.g., multiple-choice answers,acceptable answers, answers to suggest).

The question to ask 910 and potential responses 915 may be used, forexample, in accordance with various embodiments described herein foroutputting a question to a camera user. According to some embodiments ofthe present invention, if the question is a multiple choice question,then a plurality of potential answers 915 may be presented to the user.The user may then answer the question by selecting one of the potentialanswers. In another example, question identifier 905 may be used (e.g.,in conjunction with a response database) in ensuring that a camera doesnot repeat the same question.

According to at least one embodiment, a condition may comprise a Booleanexpression. This Boolean expression, for example, may reference one ormore variables (i.e., factors) and may include Boolean modifiers andconjunctions (e.g. AND, OR, XOR, NOT, NAND), comparators (e.g., >, <,=, >=, <=, !=), mathematical operations (e.g. +, −, *, /, mean, standarddeviation, logarithm, derivative, integral), functions (e.g.,search_term_in_database( ), autocorrelation( ), dilate( ),fourier_transform( ), template_match( )), and constants (e.g., 10, 20pixels, 300 milliseconds, 4 lumens, 0.02, 15%, pi, TRUE, yellow,“raining,” 5200 K). Some examples of conditions comprising Booleanexpressions include, without limitation:

-   -   (i) picture_of_football_player (image2394) AND (NOT        Date=“Halloween”)    -   (ii) (identify_person_in_image (image736234)=“FAILED”)    -   (iii) (subject_of_image (image420378)=“dog”)    -   (iv) (average_rate_of_capturing_images>2.1 per minute) OR        (empty_memory<10 Mb)    -   (v) (number_of_images>10) AND        (percentage_of_images_captured_using_flash>80%)    -   (vi) (average_audio_noise_level>=70 dB)    -   (vii) (average_rating (image45618, image86481, image18974)<5)

A condition may be based on one or more factors. Examples of factorsinclude, without limitation: factors affecting the occurrence of acondition, factors affecting whether a condition is true, factorscausing a condition to occur, factors causing a condition to becometrue, and factors affecting the output of a message.

Some general categories of factors include, without limitation: factorsrelated to images, indications by a user, time-related factors, factorsrelating to a state of the camera, information from sensors,characteristics of a user, and information from a database.

Examples of factors relating to indications by a user include, withoutlimitation: usage of controls (e.g., a shutter button, an aperturesetting, an on/off setting), voice commands (e.g., recorded by amicrophone), movement of the camera (e.g., observed using an orientationsensor), ratings provided (e.g., a user may rate the quality of an imageor how much he like an image), and responses to previous questions. Forexample, the camera may use feedback to determine the next question in aseries of questions to ask a user.

Examples of time-related factors include, without limitation: theduration of a condition (e.g., for the last ten seconds, for a total offifteen minutes), the current time of day, week, month, or year (e.g.,12:23 pm Sep. 6, 2002), a duration of time after a condition occurs(e.g., two seconds after a previous image is captured), an estimatedamount of time until a condition occurs (e.g., ten minutes until thecamera's batteries run out, twenty minutes before the sun goes down).

Some examples of factors relating to the state of the camera include,without limitation: current and past settings (e.g., shutter speed,aperture, mode), parameters that affect the operation of the camera,current and past modes (e.g., “Sports” mode, “Manual” mode, “Macro”mode, “Twilight” mode, “Fluorescent Light” mode, “Silent” mode,“Portrait” mode, “Output Upon Request” mode, “Power-Save” mode), imagesstored in memory (e.g., total images stored, amount of memoryremaining), current mode (e.g., “Sports” mode), and battery chargelevel.

Examples of factors relating to information from sensors include,

without limitation: location of the camera (e.g., determined with a GPSsensor), orientation of the camera (e.g., determined with an electroniccompass), ambient light (e.g., determined with a light sensor), soundsand audio (e.g., determined with a microphone), the range to a subject(e.g., as determined using a range sensor), a lack of movement of thecamera (e.g., indicating that the user is aiming the camera), signalsfrom other devices (e.g., a radio beacon carried by a subject, a secondcamera), and temperature (e.g., determined by a temperature/infraredsensor).

The camera may ask different questions to different types of users. Someexamples of factors relating to characteristics of a user include,without limitation: preferences for capturing images (e.g., likes highcontrast pictures, likes softening filters, saves images at best qualityJPG compression), habits when operating camera (e.g., forgets to takeoff the lens cap, turns camera on and off a lot), appearance (e.g., isthe user of the camera in an image captured using a self-timer?),characteristics of users other than the current user of the camera(e.g., past users, family members), family and friends, and skill level.For example, a skilled user may tend to capture images that arewell-composed, correctly exposed, and not blurry. In contrast, aless-experienced user may tend to have trouble capturing high-qualityimages.

As discussed variously herein, various embodiments of the presentinvention provide for information to be stored in one or more databases.Some examples of factors relating to information stored in a databaseinclude, without limitation:

-   -   (i) templates or other information useful in recognizing or        processing images    -   (ii) images stored in the camera's memory (e.g., an image        database such as the one shown in FIG. 8)    -   (iii) indications by a user (e.g., previous answers to        questions)    -   (iv) predicted weather conditions    -   (v) current weather conditions    -   (vi) topography, vegetation    -   (vii) locations of landmarks    -   (viii) light sources (e.g., all of the lights in this building        are fluorescent)    -   (ix) anticipated events (e.g., Old Faithful at Yellowstone        National Park erupting)    -   (x) the current score of a baseball game    -   (xi) sunrise and sunset times    -   (xii) high and low tide times

The camera may store a determination condition database for storinginformation related to such conditions, such as the one shown in FIG.10. For each determination condition stored in the condition database, acorresponding question may be output if the determination condition istrue. For example, QUES-123478-02 (“What kind of light bulbs does thisroom have?”) may be output to a user if the camera is indoors and theflash is turned off.

Referring now to FIG. 10, an exemplary tabular representation 1000illustrates one embodiment of the determination condition database 435(FIG. 4) that may be stored in the computing device 400. The tabularrepresentation 1000 of the determination condition database includes anumber of example records or entries, each defining a determinationcondition that may be useful in determining one or more questions to aska user of a camera. Those skilled in the art will understand that thedetermination condition database may include any number of entries.

The tabular representation 1000 also defines fields for each of theentries or records. The exemplary fields specify: (i) a determinationcondition 1005 that defines a particular determination condition, and(ii) a question to ask 1010 that includes an identifier of a questioncorresponding to the determination condition.

Question to ask 1010 preferably contains a unique reference to aquestion (e.g., as stored in a corresponding record of a questiondatabase). Of course, a determination condition database may include theactual text of a question and thus may not require a questionidentifier. For each condition, the determination condition databasepreferably stores a condition for asking at least one question and atleast one question to ask if the condition is true. As depicted intabular representation 1000, for example, “QUES-123478-02” may be outputto a user if the user is indoors and the camera's flash is turned off.According to another example, a question may be output to a user if acondition is true. For example, “QUES-123478-02” of FIG. 9 (“What kindof light bulbs does this room have?”) may be output to a user if thecamera is indoors and the flash is turned off. Note that the questionidentifier listed in the exemplary question to ask field 1010 maycorrespond to a question identifier in the question database shown inFIG. 9. For example, “QUES-123478-01” refers to the question “Are youindoors or outdoors?” in tabular representation 900.

Referring now to FIG. 11, an exemplary tabular representation 1100illustrates one embodiment of the output condition database 440 (FIG. 4)that may be stored in the computing device 400. The tabularrepresentation 1100 of the output condition database includes a numberof example records or entries, each defining a output condition that maybe useful in determining when and/or how to output a question to a user.Those skilled in the art will understand that the output conditiondatabase may include any number of entries.

The tabular representation 1100 also defines fields for each of theentries or records. The exemplary fields specify: (i) a camera mode1105, (ii) a question ready indication 1110, (iii) an output condition1115 that indicates a condition for outputting a question, (iv) a method1120 that indicates a preferred method for outputting a question, and(v) an enabled field 1130 that indicates whether the correspondingcamera mode (e.g., “Fully Automatic” mode) is presently enabled.

The output condition database stores information that may be useful indetermining when and/or how to output at least one question to a user,as discussed herein. For example, an audio recording of a question maybe output when a user presses the camera's shutter button halfway down.

As discussed herein, a mode of a camera may include a collection of oneor more parameters that affect when and/or how at least one question isoutput to a user. For example, the camera may have an “Output UponRequest” mode in which one or more questions may be output to a userwhen the user presses an “Ask Me a Question” button on the camera. Priorto outputting a question to a user, the camera may output an indicationthat a question is ready to be output. The question ready indication1110 thus describes what indication (if any) may be output to a user toindicate that the camera has a question for the user. For example, asdepicted in tabular representation 1100 of FIG. 11, if the camera is in“Sports” mode, then the camera may “beep” when it determines a questionto be output to a user. The question itself may then be output at alater time (e.g., when an output condition occurs).

As described variously herein, a question may be output to a user uponthe occurrence of one or more output conditions. For example, when thecamera is in “Manual” mode, the camera may output a question when theviewfinder is in use (i.e., the user is looking through the viewfinder).In a second example, the camera may output a question after thirtyseconds of inactivity if the camera is in “Silent” mode. The method ofoutput field indicates how a question may be output. As describedherein, a question may be output to a user in a variety of differentways. For example, a text representation of a question may be displayedon the camera's LCD screen, or an audio recording of a question may beoutput using an audio speaker.

The currently enabled field 1130 indicates whether the associated mode(i.e., the mode indicated in the “camera mode” field) is currentlyenabled. If a mode is enabled, then a question or question readyindication may be output according to the output condition and/or methodof output corresponding to that mode. If a mode is disabled, then aquestion may be output in a different manner (or may not be output atall). It is anticipated that a user may enable and disable modes basedon his preferences. For example, if a user is capturing pictures of amusical, then the user may enable “Silent” mode on the camera, so as notto disturb audience members or actors.

Of course, while the exemplary data shown in FIG. 11 indicates that onlyone mode is enabled (“Manual”), it will be understood that any number ofmodes (including no modes at all) may be enabled.

The exemplary embodiment of the output condition database shown in FIG.11 describes one example of a camera communicating with an electronicdevice. For example, when the camera is in “PDA Assisted” mode, thecamera may output a question by transmitting the question to a user'sPDA. The user may then respond to this question using the PDA (e.g., byselecting a response using the PDA's stylus) and the PDA may transmitthe user's response back to the camera.

Referring now to FIG. 12A and FIG. 12B, an exemplary tabularrepresentation 1200 illustrates one embodiment of the response database445 (FIG. 4) that may be stored in the computing device 400. The tabularrepresentation 1200 of the response database includes a number ofexample records or entries, each defining a response that may be usefulin recording responses provided by a user (e.g., in response to aquestion). Those skilled in the art will understand that the responsedatabase may include any number of entries.

The tabular representation 1200 also defines fields for each of theentries or records. The exemplary fields specify: (i) a question 1205that indicates a question that was output (e.g., to a user), (ii) a time1210 that indicates when the question was output, (iii) a response tothe question 1215 that includes an indication of the response to thequestion (e.g., text, an audio file), and (iv) an action 1220 thatindicates what (if any) actions were taken based on the correspondingresponse.

For example, the first record in the response database shown in FIG. 12Aindicates that a user responded “Indoors” to “QUES-123478-01.” Anindication of what question was output to a user may comprise a questionidentifier, for example, which may correspond to a question identifierin the question database (e.g., as represented in FIG. 9). For example,“QUES-123478-01” refers to the question “Are you indoors or outdoors?”in the question database shown in FIG. 4. As indicated in the exemplarytable, a question may be output multiple times (e.g., in differentsituations). For example, “QUES-123478-01” was output to a user at 1:34p.m. on Aug. 3, 2002 and also at 11:36 p.m. on Aug. 3, 2002.

The response database may indicate that a user has not responded to aquestion (e.g., for “QUES-123478-01” at 4:10 p.m. on Aug. 17, 2002). Inanother example, a user may have provided a response that is not ananswer to the question (e.g., for “QUES-123478-08” at 7:21 p.m. on Aug.11, 2002). Thus, in some instances, the camera may determine to outputthe question again. As discussed herein, camera may perform one or moreactions based on a user's response to a question. For example, thecamera may meta-tag at least one image or adjust one or more settings onthe camera based on a user's response to a question.

Referring now to FIG. 13A and FIG. 13B, exemplary tabularrepresentations 1300 and 1350 illustrate two embodiments of an event logdatabase 450 (FIG. 4) that may be stored in the computing device 400. Anevent log may store a list of events that occurred at one or morecameras and/or servers, for example. FIGS. 13A and 13B show two examplesof event logs that may be stored by the camera and/or a server. FIG. 13Ashows an exemplary event log for events that occurred on Aug. 3, 2002,generally relating to a user capturing images at a wedding. FIG. 13Bshows an exemplary event log for events that occurred on Aug. 10, 2002,generally relating to a user capturing images at a beach. As depicted inthe figures, event logs preferably store an indication of a time when anevent occurred and a description of the event.

Each tabular representation of the event log includes a number ofexample records or entries, each defining an event. Those skilled in theart will understand that the response database may include any number ofentries.

The tabular representation 1300 also defines fields for each of theentries or records. The exemplary fields specify: (i) a time of event1305 that indicates a time that the corresponding event occurred, and(ii) a description of the event 1310 that includes (e.g., in text) adescription of what the event was. Tabular representation 1350 definessimilar fields time of event 1355 and description 1360.

For convenience of discussion, each of the exemplary events logged hasbeen depicted as occurring at a different time. Of course, two or moreevents may be logged as occurring at the same time. Note that the eventtimes in the exemplary tables are examples only and do not necessarilyrepresent delays that may be associated with processes on the camera.For example, while the event log in FIG. 13A shows that the camerareceived a user's response to question “QUES-123478-01” at 1:35 pm onAug. 3, 2002 and then output “QUES-123478-02” at 1:36 PM on Aug. 3,2002, this does not necessarily mean that there is a one minute delaybetween receiving a user's response to question “QUES-123478-01” andoutputting “QUES-123478-02.”

The event logs depicted in FIGS. 13A and 13B may not include one or moreevents that occur at the camera. For example, the event log shown inFIG. 13A does not include an event of meta-tagging the image“WEDDING-01” as being captured indoors.

FIG. 14 shows an example of an expiring information database that may bestored by a camera, server, or other computing device. This databasepreferably stores data about information that is useful to a camera aswell as an indication of one more conditions under which thisinformation expires (e.g., and should no longer be used by the camera.)The expiring information database may also indicate one or more actionsto perform in response to information expiring.

Referring now to FIG. 14, an exemplary tabular representation 1400illustrates one embodiment of the expiring information database 455(FIG. 4) that may be stored in the computing device 400. The tabularrepresentation 1100 of the expiring information database includes anumber of example records or entries, each defining expiring informationthat may be useful in determining when information (e.g., as collectedby a camera and/or server) should expire. Those skilled in the art willunderstand that the expiring information database may include any numberof entries.

The tabular representation 1400 also defines fields for each of theentries or records. The exemplary fields specify: (i) the information1405 whose expiration is being monitored, (ii) an expiration condition1410 that indicates when or under what circumstances the piece ofinformation should expire, and (iii) an action 1415 that indicates anaction (if any) to be performed (e.g., by a camera, by a server) inresponse to the information expiring.

For example, the first record in the tabular representation 1400 of theexpiring information database shown in FIG. 14 indicates that the camerawill disregard the information that the camera is outdoors and outputthe question, “Are we outdoors?” if the camera is turned off for morethan sixty minutes.

The information field 1405 preferably includes the piece of information(e.g., determined based on a user's response to a question). Forexample, the camera may store the information “Camera is Outdoors” basedon a user responding “outdoors” to the question “Are you indoors oroutdoors?” QUES-123478—The expiration condition 1410 preferablyindicates one or more conditions under which the information willexpire. For example, the information that the weather outside is sunnymay be set to expire if the sun goes down or if the camera is takenindoors.

When an expiration condition 1410 occurs, the camera may perform one ormore actions. Exemplary actions include outputting a question, adjustinga setting, or ceasing to perform an action (e.g., an action that wasperformed based on the information being current). For instance, thecamera may cancel “Sunny Beach Mode” (e.g., automatically or afterprompting the user) in response to the expiration of the informationthat a user is on a beach.

2. Processes

Methods consistent with one or more embodiments of the present inventionmay include one or more of the following steps, which are described infurther detail herein:

-   -   (i) capturing an image,    -   (ii) determining a question,    -   (iii) outputting a question,    -   (iv) receiving a response to a question,    -   (v) performing an action based on a response, and    -   (vi) expiring information.

Referring now to FIG. 15, a flowchart illustrates a process 1500 that isconsistent with one or more embodiments of the present invention. Theprocess 1500 is a method for outputting a question to a user of acamera. The process 1500, and all other processes described hereinunless expressly specified otherwise, may be performed by an imagingdevice (e.g., a camera), a computing device (e.g., a server) incommunication with an imaging device, and/or a combination thereof. Eachof these devices is described in detail herein. Further, the process1500, and all other processes described herein unless expresslyspecified otherwise, may include steps in addition to those expresslydepicted in the Figures or described in the specification, withoutdeparting from the spirit and scope of the present invention. Similarly,the steps of process 1500 and any other process described herein, unlessexpressly specified otherwise, may be performed in an order other thandepicted in the Figures or described in the specification, asappropriate.

Referring to step 1505, an image is captured. Various ways of capturingan image, including by use of a camera, are well known to those skilledin the art and some examples are provided herein. In step 1510, aquestion is determined based on the captured image (e.g., using adetermination condition database of a camera). In step 1515, thedetermined question is output to a user (e.g., via an output device of acamera). Various ways of determining and of outputting questions aredescribed in detail herein.

Referring now to FIG. 16, a flowchart illustrates a process 1600 that isconsistent with one or more embodiments of the present invention. Theprocess 1600 is a method for performing an action based on a responsefrom a user. For illustrative purposes only, the process 1600 isdescribed as being performed by a camera 130. Of course, the process1600 may be performed by any type of imaging device 210, or an imagingdevice 210 in conjunction with a computing device 220.

Referring to step 1605, a camera 130 captures an image. For example, auser presses a shutter button to record an image of a scene. In anotherexample, the camera 130 automatically captures an image of a scene(e.g., in order to make suggestions that the user adjust one or moresettings). In step 1610, the camera determines a question based on theimage. For example, the camera 130 may determine that the image isunderexposed and may determine that it is appropriate to ask the user ifthe user intended the image to be underexposed. In another example, thecamera 130 may transfer the image or information about the image to aserver 110 for determination of a question. Determining the question maythus include receiving an indication of a question from the server 110.

In step 1615, the camera 130 outputs the question to a user (e.g., viaan LCD device). In step 1620, the camera 130 receives a response fromthe user. As discussed herein, the user may provide a response by any ofa variety of means, including by making a selection on a displayed menuof possible responses to the question. In step 1625, the camera 130performs one or more actions based on the received response. Forexample, based on a response that the user intends the image to beoverexposed, the camera 130 may store an indication (e.g., in a responsedatabase) that questions about exposure should not be output for imagesof this scene. Various steps of exemplary processes 1500 and 1600 aredescribed in detail below.

Images may be captured (e.g., using an imaging device 210) in a varietyof ways. For example, an image may be captured based on an indicationfrom a user. For instance, a user may operate a control on a camera(e.g., a shutter button) to capture an image. An image may be capturedbased one or more settings. For example, an image that is captured by acamera may depend on the current aperture, shutter speed, zoom, focus,resolution, and compression settings. Similarly, an image may becaptured based on a current mode of the camera (e.g., “Sunset” mode,“Sunny Beach” mode). For example, the camera may have a “Sunset” mode,which describes settings that appropriate for capturing images ofsunsets.

Alternatively, an image may be captured automatically (e.g., without anyindication from a user). For example, the camera may capture images andstore them in a buffer even if a user has not pressed the shutter buttonon the camera. In order to save memory space, images that are capturedautomatically may be automatically deleted or overwritten. Capturing animage automatically may be particularly helpful in some embodiments fordetermining the subject(s) of an image the user wishes to record or howa user is composing a photograph. For example, before a user presses theshutter button on the camera, he may aim the camera at a scene (e.g.,his girlfriend in front of the Golden Gate Bridge). The camera maycapture an image of this scene and then output a question to the userbased on the image. In this manner, an image may be captured and aquestion may be output to a user prior to the user actually taking apicture.

Capturing an image may include capturing an image based on a condition.This condition may be referred to herein as a capture condition todifferentiate it from other described conditions.

Those skilled in the art will readily understand that capturing an imagemay include storing the image in memory. As discussed herein, variousdifferent forms of memory may be used to store an image, including,without limitation: non-volatile memory (e.g., a CompactFlash™ card),volatile memory (e.g., dynamic random access memory (DRAM) forprocessing by an image recognition process), removable memory (e.g., aSmartMedia™ or CompactFlash™ card), and non-removable memory (e.g., aninternal hard drive). Images may be stored in an image database, such asthe one shown in FIG. 8.

Other methods and aspects of capturing an image (e.g., using a digitalcamera) are known to those skilled in the art and need not be describedin detail herein.

In accordance with various embodiments of the present invention, acamera and/or server may output various different types of questions toa user. Such questions to ask a user may be determined (e.g., by aserver) in many different ways, as discussed variously herein by way ofexample and without limitation. For instance, questions may bedetermined based on a condition, based on an image, and/or based on atemplate. A camera may determine a question to ask a user based on avariety of different factors, including images stored in the camera'smemory, the state of the camera (e.g., the camera's current settings),indications by a user (e.g., responses to previous questions), andinformation from sensors (e.g., information captured by an imagesensor).

For example, if a user captured an image of a group of people, then thecamera may use image recognition software to determine that this imagecorresponds to a group of people and ask the user a question about thisgroup of people (e.g., “Who's in this picture?”). According to oneembodiment, the camera may determine a question based on an image thatwas captured automatically (i.e., without the user pressing the shutterbutton on the camera).

According to one or more embodiments of the present invention, a cameramay communicate with a server in order to determine a question to outputto a user. For example, the camera may transmit any of variousinformation (e.g., images, GPS coordinates) to a computer server. Theserver may then determine one or more questions based on thisinformation. The server may then transmit an indication of at least onequestion to the camera, and the camera may output the question to auser.

Various types of information that may be collected by a camera aredescribed herein. Some examples of information that a camera maytransmit to a server include, without limitation: one or more imagescaptured by the camera, indications by a user (e.g., responses toquestions, usage of controls), a state of the camera (e.g., currentmode, images stored in memory), and information from sensors (e.g.,location, orientation, sound and audio).

A server may determine a question to output in accordance with one ormore of the exemplary processes discussed herein. It is worthwhile tonote that the computer server may have significantly greater processingpower, memory, power consumption (e.g., no batteries), and physical size(e.g., not portable) than the camera. This may allow the computer serverto perform computations and analysis that are more complex or extensivethan could those that could be performed quickly using the camera'sprocessor. For example, the computer server could run complicated imageanalysis and pattern matching algorithms to determine an appropriatequestion to ask a user.

After a question is determined, a computer server may transmit anindication of the question to the camera. Examples of indications ofquestions include, without limitation, a question and a questionidentifier. For example, the computer server may transmit an audio clipor text messages corresponding to the question. The question may becompressed (e.g., as an MP3) to reduce the bandwidth necessary totransmit the question. In another example, the camera may store adatabase of questions, with each question in the database beingidentified by a question identifier. In order to indicate a question tothe camera, the computer server may transmit a question identifiercorresponding to the question. The camera may then retrieve the questionfrom the database.

After receiving an indication of a question from the computer server,the camera may output this question to a user as discussed variouslyherein.

According to some embodiments, a camera may determine a question to aska user based on a condition. This type of condition may also be referredto as a determination condition to differentiate it from otherconditions described elsewhere in this disclosure.

Some examples of determining a question based on a condition arediscussed herein, without limitation. For instance, if the batteries inthe camera are running low (i.e., a condition), then the camera may askthe user, “How many more pictures are you planning on taking?” In oneexample, if a captured image includes an image of a football player(i.e., a condition), the user is to be asked, “Are you taking picturesof a football game?” In another example, if a captured image includes animage of a candle (i.e., a condition), the user may be asked, “Are youtaking pictures of a birthday party?” In one example, if a capturedimage includes an image of two or more people (i.e., a condition), thenthe question, “Who are the people in the photo?” is to be output to theuser.

As discussed variously herein, in accordance with some embodiments ofthe present invention, a camera or server may determine a question toask a user based on one or more images that have been captured. That is,an image may be captured and then the camera may determine a question toask a user based on this image.

As discussed herein, determining a question based on an image mayinclude processing the image using image recognition software. A widevariety of image recognition programs are known to those skilled in theart and need not be described in detail herein.

In one example of determining a question based on an image, a usercaptures an image using the camera. The camera may determine, based onan analysis of the image, that this image shows a person sitting under atree. Based on this determination, the camera may ask the user aquestion such as, “Who is the person sitting under the tree in thispicture?” The user's response to this question (e.g., “Alice”) may beused, for example, to meta-tag the image.

In another example, the camera may automatically capture a plurality ofimages of an exciting series of plays during a basketball game. Based onthese images, the camera may ask the user, “Are you interested inpictures of any of the following players (check all that apply)?” Thecamera may then save or delete some or all of the captured images basedon the user's response to this question.

In another example of determining a question based on an image, a usermay capture an image of a child with a present. Based on this image, thecamera may ask the user, “Are you taking pictures of a birthday party?”If the user indicates that he is indeed taking pictures of a birthdayparty, the camera may adjust the shutter speed to be at least 1/125 sec,so as to capture the children who are moving quickly.

The camera may determine a question based on one or more properties ofan image, including, without limitation: exposure (e.g., includingbrightness, contrast, hue, saturation), framing (e.g., organization ofsubjects within the image, background), focus (e.g., sharpness, depth offield), digitization (e.g., resolution, quantization, compression),meta-information associated with an image (e.g., camera settings,identities of people in an image, a rating provided by a user),subject(s) (e.g., people, animals, inanimate objects), scenery (e.g.,background), and motion relating to an image (e.g., movement of asubject, movement of the camera).

The camera may determine a question based on exposure of an image. Forexample, the camera may determine that the background of an image isbrighter than the foreground of the image. Based on this, the camera mayask a user, “Would you like to use a fill flash to brighten yoursubject?” In another example, the camera may determine that an image istoo bright. Based on this, the camera may ask a user, “Are you trying totake a picture of a sunrise or sunset?”

The camera may determine a question based on framing of an image. Forexample, the camera may determine that an image includes two objects,one in the foreground and one in the background. Based on this, thecamera may ask a user, “Which object do you want to focus on, the one inthe foreground or the one in the background?” If the objects have beenidentified (e.g., by the camera using an image recognition program), thedifferent objects may be named in the question.

In an example of how camera may determine a question based on focus ofan image, the camera may determine that a portion of an image is blurred(e.g., as if by movement). Based on this, the camera may ask a user,“Are you taking pictures of a sporting event?” Based on the user'sresponse, the camera may then adjust a setting on the camera (e.g.,increase the shutter speed to at least 1/250 sec) as described herein.

The camera may determine a question based on meta-information associatedwith an image (e.g., camera settings, identities of people in an image).In one example, the camera recognizes that an image has been meta-taggedas being taken at 7:06 p.m. Based on this information, the camera mayask a user, “Is the sun about to go down?” or “How long will it be untilthe sun goes down?” In another example, an image may include a meta-tagthat indicates that it shows Alice and Bob in a canoe. Based on thistag, the camera may ask a user, “Are you taking pictures at a lake orriver?” In another example, an image may include a meta-tag thatindicates the preferred cropping or scale for the image. Based on this,the camera may ask a user, “When your subject is off-center, do you wantthe background to be in focus?” In still another example, meta-dataassociated with an image may include a rating of the quality of theimage (e.g., a rating provided by the user or determined by the camera).The camera may determine a question based on this rating (e.g., “Thisimage appears to be overexposed. Are you trying to create an artisticeffect?”).

As discussed herein, the use of image recognition software may allow forone or more subjects of a captured image to be determined. Examples ofsubjects include, without limitation: people, animals, buildings,vehicles, trees, the sky, a ceiling, a landscape, etc. Determining aquestion may comprise determining a type of scenery (e.g., naturallandscape) in the image. Scenery may include one or more subjects, whichmay or may not be identified individually. Further, according to someembodiments, one or more questions may be determined based on whetherthe image matches a template.

In one example of determining a question based on a subject in an image,the camera may identify a candle in an image. Based on thisdetermination, the camera may ask a user, “Are you taking pictures at abirthday party?” In another example, the camera may identify a largebody of water in an image. Based on this, the camera may ask a user,“Are you on a boat?” Based on a determination that an image includes abuilding, for example, the camera may ask a user, “Are you outside?” Ifit is determined that an image includes a mountain, for example, thecamera may ask a user, “What mountain is this?”

It will be readily understood that at least one subject of an image maybe a person. In some embodiments, determining a question may include oneor more of the following steps: determining that an image includes atleast one person, identifying at least one person in an image, anddetermining one or more characteristics of at least one person in image.One or more of the above steps may be performed by a server and/orcamera.

In one example of determining a question based on a subject in an image,the camera may identify a person in an image (e.g. based on informationreceived from a server). Based on this identification, the camera mayask a user, “Is this a picture of Alice?” The camera may determine aquestion based on a person in an image. In another example, the cameramay ask a user, “Do you already have any pictures of this person?” Acamera may ask a user, “What color skin does Bob have?” The user'sanswer to this question may be useful in determining how to set exposuresettings on the camera when taking a picture of Bob. The subject of animage may be identified as a football player and thus may indicate thata user is capturing images of a football game. Accordingly, the cameramay ask the user, “Are you taking pictures of a football game?”

The camera may determine a question based on motion relating to an image(e.g., based on blurring of an image or comparison of a plurality ofimages). For example, the camera may determine a question based onmotion of a subject in an image. For instance, if a subject in an imageis moving quickly, the camera may ask a user, “Are you taking sportspictures?” In another example, the camera may determine that the groundplane in an image is moving slightly (e.g., shimmering like water).Based on this, the camera may ask a user, “Are you taking pictures ofwater (e.g., the ocean, a fountain, a creek)?” An imaging device maydetermine that it is moving (e.g., using a GPS sensor). Based on thisdetermination, a digital camera may ask a user, “Are you in a vehicle(e.g., a car, a boat, an airplane)?”

Various embodiments of the present invention allow for a question to bedetermined based on a plurality of images. For example, the camera maycapture two images in close succession (e.g., 1/10 of a second apart).The camera may compare these images and may determine that a subject(e.g., a person, an animal) is moving. Based on this determination, thecamera may ask a user, “Are you taking pictures of a sporting event?” Insome embodiments related to multiple images, a camera may determine aquestion based on at least one difference among a plurality of images.For example, the camera may capture a plurality of images in brightlight conditions (e.g., outdoors on a sunny day). Then the camera maycapture an image in low light conditions (e.g., so little light that aflash in necessary). Based on this, the camera may ask a user aquestion, “Did you just go inside a building?”

The camera may also be configured so as to determine a question based onat least one similarity among a plurality of images. For example, thecamera may determine that two images have the same person wearing a redshirt in them. Based on this, the camera may ask a user, “Who is theperson in the red shirt?” In another example, the camera may determinethat a first image is of a tiger and a second image is of a polar bear.Since a tiger and a polar bear are both animals, the camera may ask auser, “Are you at the zoo?”

A question may refer to an image (and thereby be based on the image).Examples include, without limitation:

-   -   (i) “Is this picture bright enough?”    -   (ii) “Who is the person wearing the yellow jacket in this        image?”    -   (iii) “Do you mind if I delete this image? You have two others        just like it.”    -   (iv) “Where was this picture taken?<show the picture>?”    -   (v) “Please highlight Alice's face in this image.”

According to at least one embodiment of the present invention, one wayof determining a question based on an image is to determine if an imagematches a template. For example, a camera and/or server may store aplurality of templates. Each template may correspond to a different typeof image, category of image, or one or more properties of an image.After an image is captured, this image may be compared to one or more ofthe templates to see if there is a match. If the image matches atemplate, then an appropriate question may be determined (e.g., toverify that the image does in fact match the template and/or to verifyinformation useful in determining a setting on the camera).

Some examples of templates include, without limitation:

-   -   (i) An “indoors” template. If an image matches with the indoors        template, then the camera may determine that there is a        significant probability that the user is capturing images inside        a building. The camera may then determine an appropriate        question to ask the user (e.g., “Are you indoors?”).    -   (ii) A “sandy beach” template. If an image matches with the        “sandy beach” template, then the image may have been captured on        a sandy beach (e.g., in bright sunlight, where water may be        nearby). The camera may then determine an appropriate question        to ask the user (e.g., “Are you at the beach?”).    -   (iii) A “football player” template. If an image matches the        “football player” template, then the image may include a        football player. The camera may then determine to ask the user,        “Are you taking pictures of a football game?” Note that in a        preferred embodiment a football player template may match with        any picture of a football player, whether the player is facing        the camera, facing away from the camera, or lying on the ground.        Alternatively, there may multiple “football player” templates,        with some corresponding to a football player in a different        position, with different lighting, etc.    -   (iv) A “candle” template. If an image matches the “candle”        template, then the image may include a candle or other point        light source (e.g., light bulb, flashlight). Based on this, the        camera may determine a question to ask a user (e.g., “Are you        taking pictures at a birthday party?”).    -   (v) A “Bob Jones” template. In an image matches the “Bob Jones”        template, then the image may include a picture of Bob Jones        (e.g., who may be a friend of the user of the camera). Based on        this, the camera may determine a question to ask a user (e.g.,        “What is Bob doing in this picture?”).    -   (vi) A “fluorescent light bulb” template. If an image matches        with this template, then the scene in the image may have been        illuminated with a fluorescent light bulb. The camera may ask a        user an appropriate question based on the image and the        determined template.

In accordance with various embodiments of the present invention, acamera and/or a server may perform one or more of the following steps inmatching an image to a template:

-   -   (i) determining an image    -   (ii) storing one or more templates (e.g., in a template        database)    -   (iii) determining that the image matches at least one template    -   (iv) determining a correspondence between the image and at least        one template    -   (v) determining a question based on the image and the at least        one template    -   (vi) determining a correlation between the image and at least        one template    -   (vii) determining a degree to which the image matches at least        one template

Note that the matching of an image to a template may be a condition.That is, if an image matches a template, then a condition may be trueand a question may be determined based on this condition, as describedherein.

It will be understood that an image may match with multiple templates.For example, a picture of Bob Jones on a beach may match with both the“Bob Jones” template and the “sandy beach” template. In thiscircumstance, a question may be determined based on one or both of thetemplates.

Also, it may be possible for an image to partially match a template. Forexample, matching an image to a template may include determining howmuch the image matches the template. Some examples of partial matchesinclude, without limitation:

-   -   (i) An image of a light bulb may only be a 65% match for the        “candle template.”    -   (ii) An image may match a first template to a first degree and a        second template to a second degree. For example, an image of        Steve Jones (who looks similar to his bother Tom Jones) may be a        95% match with the “Steve Jones” template and a 80% match with        the “Tom Jones” template.    -   (iii) An image that was taken indoors may be a 5% match for the        “sandy beach” template.    -   (iv) An image may only be considered a match for a template if        the amount that the image matches the template is greater than a        threshold value (e.g., 95% certainty).

According to some embodiments, the camera may determine and/or create atemplate based on a user's response to a question. For example, thecamera may create an “Alice Jones” template based on an indication by auser that an image is of Alice Jones.

After a question is determined, a camera may output the question to auser. Various different ways that a question may be output to a user ofa camera are discussed herein.

A question may be output to a user of a camera in a variety of differentways. For example, a question may be output using one or more of avariety of different output devices. Examples of output devices that mayoutput questions to a user include, without limitation:

-   -   (i) An LCD screen. For example, the camera may include a color        LCD screen on its back. This LCD screen may display questions to        a user (e.g., as text).    -   (ii) An audio speaker. For example, the camera may use an audio        speaker to output a audio clip of a question to a user.    -   (iii) An LED screen. For example, the camera may include a LED        screen that displays questions to users as scrolling text.    -   (iv) A heads-up viewfinder display. For example, a question may        be displayed on the viewfinder of the camera so that a user can        view the question while composing a shot (i.e., which the user        is preparing to capture an image).    -   (v) A printer. For example, the camera may include thermal        printer that may print out a list of questions for a user to        answer.        Other examples of output devices that may be used to output a        question are discussed herein.

A question may be represented in or more of a variety of different mediaformats, including text, audio, images, video, and any combinationthereof. Examples of questions being output as text include a questiondisplayed as text on an LCD screen on the back of the camera and aquestion displayed as text overlaid on the camera's viewfinder. Inanother example, a text question may repeatedly scroll across a headerbar on the camera's LCD screen. It will be understood that variousvisual cues may be used to draw a user's attention to a message that isoutput in text, including different fonts, font sizes, colors, textboxes, and backgrounds.

Examples of questions being output as audio include an audio recordingof a question. In another example, speech synthesis software may be usedto generate an audio representation of a question. In another example, a“BEEP” sound may be output when a question is displayed on a videoscreen. Examples of outputting questions using images and video include,without limitation, displaying a sequence of images (e.g., a movie) to acamera user using a video screen. In another example, a video of ananimated cartoon character may indicate a question to user.

Note that a message may be presented in a plurality of ways. Forexample, a question may include both a text component and an audiocomponent (e.g., the camera may beep and then display a question on anLCD screen). In a second example, the camera may display an image on itscolor LCD screen and then play an audio recording of a question.

Note that a question may be phrased in the first person. For example, aquestion may use the word “I” to refer to itself or the word “we” torefer to the camera and the user. Examples include, without limitation:

-   -   (i) “Are we at the beach?”    -   (ii) “It seems to me that we're indoors right now. Is this        correct?”    -   (iii) “Am I underwater?    -   (iv) “How many more pictures should I plan on taking today?”    -   (v) “Should I use the red-eye reduction flash?”

A question may be output in different languages (e.g., depending on whois using the camera). For example, if the current user of the cameraspeaks English, then a question may be output to the user in English.However, if the current user of the camera speaks Chinese, then thequestion may be output the to the user in Chinese.

According to some embodiments of the present invention, a question maybe output by a presenter (e.g., a character that presents the questionto a user). Examples of presenters include, without limitation

-   -   (i) A speaker. For example, the camera may store two recordings        of a question—one with a female speaker and one with a male        speaker.    -   (ii) An animated character in a video message. For example, an        avatar, virtual assistant, or other on-screen character may be        displayed to a user in conjunction with a question. For example,        an animated rabbit may be displayed on the camera's LCD screen        and “talk” to a user, thereby outputting one or more questions        to the user. Indications from the rabbit may be provided as text        (e.g., displayed using a speech bubble as a partition) and/or        audio (e.g., an audio recording may be played, allowing the        rabbit to “speak” to the camera user.)    -   (iii) An actor. For example, a video of an actor presenting a        question may be displayed to a user on the camera's LCD screen.        The camera may store a database of video clips representing        different questions.    -   (iv) A celebrity. For example, an audio or video recording of a        celebrity (e.g., William Shatner) reciting a question may be        output to a user.

It is anticipated that some types of camera users may pay more attentionto representations of message that include certain presenters. Forexample, a user may pay extra attention to a message that is presentedby his favorite celebrity.

In order to accommodate a variety of different formats and languages forthe same question, the camera may store one or more representations of aquestion in memory. For example, for a given question (e.g., “Are you ata ski resort?”), the camera may store the following representations: atext version of the question in English, an audio version of thequestion in English, a text version of the question in Spanish, and anaudio version of the question in Spanish.

As discussed variously herein, a question may be output to a user at avariety of different times. For example, the camera may delay outputtinga question until an appropriate time. In another example, the camera mayoutput a questions based on a condition.

In addition to outputting a question, the camera may output otherinformation that may be helpful to the user. Examples of additionalinformation may include, without limitation: at least one image thatrelates to the question, potential answers to the question (e.g., for amultiple choice question), a reason for asking the question, currentsettings on the camera, a default answer to the question, and a categoryfor the question.

As discussed herein, the camera may output a question to a user thatrelates to at least one image. To assist the user in answering thequestion, the camera may display at least one image to the user. Forexample, the camera may display a picture of a person on the beach to auser and ask the user, “Is this a picture of Alice or Bob?” In anotherexample, the camera may display a set of four pictures to a user and askthe user, “All of these pictures are of Alice, right?”

In another example, the camera may display a plurality pictures to auser and ask the user, “Pick your favorite picture from this group.” Acamera may display an image to a user and ask the user, “Is this pictureunderexposed?” or may highlight a portion of an image ask a user, “Isthis the subject of the image?” In yet another example, the camera mayautomatically crop an image and ask a user, “I'm going to automaticallycrop this image. Is this a good way to crop the image?”

Indicating potential answers to a question may be helpful in describingto a user how he should answer a question, indicating acceptable answersto a question, and/or reducing the time or effort required for a user toanswer a question. Examples of potential answers to questions include,without limitation:

-   -   (i) “Yes”/“No”/“I'm not sure”    -   (ii) “Alice”/“Bob”/“both Alice and Bob”/“neither Alice nor Bob”    -   (iii) “tungsten”/“fluorescent”/“I'm not indoors”/“I don't know”    -   (iv) “overexposed”/“underexposed”/“just right”

Indicating at least one reason for asking a question may be helpful inexplaining the purpose of a question to a user, or in explaining to auser why it would be beneficial for him to answer a question. Someexamples of the camera indicating one or more reasons for asking aquestion include:

-   -   (i) “I thought I saw a football player in that last photo you        took. Are you taking pictures of a sporting event?”    -   (ii) “Based on the last picture you took, it looks to me like        you're on a beach. Are you taking picture on a beach?”    -   (iii) “It looks like you're taking a lot of pictures outdoors        with white backgrounds. Are you at a ski resort?”    -   (iv) “The camera seems to be rocking back and forth. Are you on        a boat?”    -   (v) “It looks like there was a reflection in the last picture        you took. Are you trying to take pictures through a glass        window?”    -   (vi) “How many more pictures are you planning on taking? If you        continue storing your pictures at high resolution, you only have        space for 12 more pictures.”

Indicating at least one setting on the camera may help the user tounderstand the context of a question, or to make a decision on how torespond to a question. In one example, the camera indicates: “The flashis currently off. Are we indoors?” In another example, the cameradisplays: “Pictures are currently being captured at 1600x1200resolution, meaning that I have enough memory left to hold 15 morepictures at this resolution. How many more pictures are you planning ontaking?” Providing such additional information may be beneficial to sometypes of users.

Some embodiments of the present invention provide for a default orpredetermined answer to a question output to a user. In some cases, thedefault answer may be indicated to the user. Some examples of outputincluding default answers to questions include:

-   -   (i) “We're indoors, right? If you don't answer in 5 seconds,        I'll assume that we're indoors . . . ”    -   (ii) “Let me know if any of this information is incorrect:        You're taking pictures at a birthday party. There are 8 children        at the birthday party. So far you've taken pictures of 3        children: Alice, Bob, and Carly.”    -   (iii) “The girl wearing the blue sweater is Alice, right? Press        the “no” button if this is incorrect.”

Questions may be categorized, allowing the camera to output anindication of at least one category corresponding to a question.Categorizing a question may be helpful to some users, for example, ifthere are a plurality of questions that may be output to a user and theuser would like to sort these questions. Questions may be categorizedbased on a variety of different factors, including, without limitation:

-   -   (i) topic (e.g., “Lighting Questions,” “Focus Questions,”        “Meta-Tagging Questions,” “Situational Questions,” “Questions        about Future Plans,” “Questions About Past Images”)    -   (ii) image (e.g., “Questions About Image #1,” “Questions About        The Last 8 Images Captured,” “Questions about Images Captured        Yesterday”)    -   (iii) priority (e.g., “High-Priority Questions,” “Questions to        Answer in the Next 5 Minutes,” “Questions to Answer During Your        Free Time,” “Questions to Answer Before Capturing More Images”)    -   (iv) type of response (e.g., “Yes/No Questions,” “Free Answer        Questions,” “Multiple Choice Questions”)    -   (v) potential actions based on a user's response (e.g.,        “Meta-Tagging Questions,” “Settings Questions,” “Image Storage        Questions”)

As discussed herein, the camera may determine a question to output to auser (e.g., based on a determination condition). Alternatively, or inaddition, the camera may determine when and/or how to output a questionto a user based on one or more conditions. For example, a question maybe output as an audio recording while a user is looking through theviewfinder of the camera. In another example, the camera may delayoutputting a question to a user until there is a pause in the user'sactivities.

The camera thus may output a question based on a condition. Thiscondition may also be referred to as an output condition todifferentiate it from other types of conditions (e.g., determinationconditions) described elsewhere in this disclosure.

According to some examples, without limitation, a question may beoutput: when a condition occurs, when a condition is true, when acondition becomes true, in response to a condition, in response to acondition occurring, in response to a condition being true, because of acondition, because a condition occurred, because a condition is true,according to a condition, at substantially the same time that acondition occurs, and/or at substantially the same time that a conditionbecomes true.

Note that conditions may be useful in enabling a variety of differentfunctions. For example, as discussed herein, a condition may be used indetermining what question(s) to output and/or for determining an orderin which to output a plurality of questions.

In another example, a condition may be useful for determining when tooutput a question (e.g., determining an appropriate time to output aquestion). According to some embodiments, output of a question may bedelayed until a condition occurs. For example, it may be annoying tooutput a question to a user when the user is busy taking photographs ata birthday party or busy talking with a friend. Therefore, outputting aquestion to the user may be delayed until an appropriate time.

In some embodiments, as described herein, a condition may be used indetermining how to output a question. For example, a condition may beused to determine whether a question should be output in text on thecamera's LCD display or as audio through the camera's audio speaker. Ina second example, the camera may select which personality should be usedin outputting a question to a user.

In light of the present disclosure, one skilled in the art willunderstand that conditions for outputting a question may be similar toconditions for determining a question. For example, it should be clearto the reader that an output condition may be a Boolean expressionand/or may be based on one or more factors

Many types of factors are discussed herein as being potentially usefulin making one or more of various types of determinations. For example,any of the factors described herein as potentially useful in determininga question to ask a user (e.g., images, indications by a user, movementof the camera, time-related factors, state of the camera, informationfrom sensors, characteristics of a user, information from a database)may also be used for determining when and/or how to output a question toa user. Still other factors will be readily apparent to those skilled inthe art in light of the present disclosure.

Some moments may be particularly appropriate for outputting a questionto a user. Some general examples of appropriate times to output aquestion include, without limitation: when a user is composing a shot(e.g., is about to capture an image), when a user is inactive, when auser indicates that he is interested in receiving at least one question,when a user is viewing an image, when a user is answering one or morequestions, when the camera's resources are running low, and when a userstarts to capture images of a scene. Some of these general examples arediscussed in further detail herein.

According to some embodiments, a question may be output when it isdetermined (e.g., by the camera) that a user is composing a shot. Someexemplary scenarios include, without limitation:

-   -   (i) Operation of a control. For example, if the user presses the        shutter button halfway down, then this may indicate that he is        composing a shot. This may be an appropriate time to ask him a        question about the shot. In a second example, the camera may        output a question to a user when the user switches the camera        from manual exposure mode to auto-exposure mode.    -   (ii) Use of the optical viewfinder on the camera. For example,        the camera may include a sensor that determines when the user is        looking through the optical viewfinder on the camera.    -   (iii) Use of the digital viewfinder on the camera. For example,        if a user turns on the digital viewfinder on his camera, this        may be a sign that he is about to start capturing images.    -   (iv) When the camera is being held steadily (e.g., in a        horizontal position). If a user is holding the camera steady in        a horizontal position, for example, it may be likely that he is        about to capture an image.    -   (v) Holding of the camera in two hands. For example, the camera        may include one or more touch sensors (e.g., heat, continuity,        electric field, pressure) that may determine when a user places        both of his hands on the camera. This may be considered an        indication that the user is composing a shot.

Some factors may be related to an indication by a user. For example, thecamera may include a button or menu option labeled “Ask Me a Question.”Whenever a user has free time, he may press this button to answer anyquestions that the camera has. An indication that the user has pressedthe button may trigger the camera to provide one or more questions. Inanother example, whenever the camera has a question to ask a user, itmay output an indication of this (e.g., by beeping once or illuminatingan LED). The user may then respond to this indication at his leisure(e.g., once he finishes capturing a sequence of action photos) byproviding an indication (e.g., pressing a button on the camera) when heis ready to answer the question.

In some embodiments, a user may provide an indication of what questionhe would like to answer. For example, a user may select a question orquestion topic from a list of questions or question topics. In a secondexample, a user may use the camera to view images that he has alreadycaptured. Some images may have questions associated with them, and toindicate this, these images may be highlighted by the camera, forexample, with green borders. To indicate what question he would like toanswer, the user may select (e.g., using a touch screen, using a dial)one or more of the highlighted images.

A user may provide an indication of how a question should be output. Forexample, the camera may normally output questions in an audio format. Auser who is operating the camera in an opera house, however, may preferto avoid disturbing other audience members. Therefore, the user mayoperate a control on the camera to indicate that he would prefer thatthe camera output the question in text form (e.g., via the camera's LCDdisplay).

Some factors relating to inactivity by a user may be used in determininghow and/or when to output a question. Such factors may include, withoutlimitation:

-   -   (i) Duration since activity by a user. For example, the camera        may include a timer that monitors the period of time that has        elapsed since a user performed an activity (e.g., operated a        control on the camera). If a threshold amount of time has        elapsed (e.g., sixty seconds), then the camera may determine        that a question should be output to the user.    -   (ii) Measurements relating to images captured by a user. For        example, the camera may monitor a duration since an image has        been captured (e.g., sixty-five seconds) or an average rate of        capturing images (e.g., one picture every thirty-two seconds).    -   (iii) Lack of sound. For example, the camera may include a        microphone that monitors the level of audible background noise        around the camera. If the level of background noise falls below        a threshold level, the camera may determine that this is an        appropriate time to output a question to a user.    -   (iv) Movement of the user and/or the camera. For example, the        camera may include one or more motion sensors that may be        helpful in determining whether the user is currently composing a        shot, moving to a new location, or otherwise engaged in an        activity. If the camera determines that the user is not moving        the camera, a question may be output the user.    -   (v) Power down. For example, if the camera determines (or a user        indicates) that the camera should enter power-save mode, then        the camera may determine that this is an appropriate time to        output a question to a user. In a second example, the camera may        output an indication that one or more questions are ready any        time the user turns the camera on or off.

Examples of factors relating to the camera outputting an image include,without limitation:

-   -   (i) Viewing an image using the camera. For example, the camera        may include a color LCD display that allows a user to view        images that are stored in the camera's memory (e.g., images that        he has already captured using the camera). If the user uses the        LCD display to view one or more images, this may indicate that        the user is no longer busy with other activities and it may be        an appropriate time to output a question to the user. For        example, the camera may output any questions relating to an        image when the user views that image on the camera.    -   (ii) Printing an image. For example, the camera may include a        printer or be connected to a printer. When a user operates the        camera to print one or more images, the camera may output one or        more questions to the user (e.g., questions relating to the one        or more images).    -   (iii) Transferring an image to another device. For example, the        camera may transfer images to one or more various other devices        (e.g., a desktop computer using a USB cable, an iPod™ portable        wallet, a television set for viewing, a color inkjet printer for        printing, a removable flash memory card for storage). One or        more questions may be output to a user before, during, and/or        after the transfer of an image to another device.

In some embodiments, it may be determined that it is appropriate to aska question when the user is answering one or more other questions. Forexample, a plurality of questions may be output to a user simultaneouslyor in close succession. For example, it may be most convenient for auser to answer all or most of the camera's questions at one time, ratherthan individually as the camera determines questions to ask the user.

In another example, a second question may be output to a user based on auser's response to a first question. For instance, the camera may ask auser, “Where are you in this picture?” If the user responds, “on abeach,” then the camera may ask the user a second, related question:“Are these two pictures taken on the same beach?”

A determination that the camera's resources are running low may be usedin determining when or how to provide a question. For example, if acamera's batteries are running low, the camera may output the question,“The batteries are running low. Do you have any more batteries?” Inanother example, if a memory resource (e.g., a flash memory card)approaches or is below a predetermined threshold of available memory(e.g., ten Mb) the camera may output the question, “You have only 10 Mbof memory left. How many more images are you planning on capturing?”

It may be desirable to output one or more questions when a user isstarting to capture images of a scene. Some exemplary related factorsand scenarios include, without limitation:

-   -   (i) A user turns the camera on. For example, each time a user        turns the camera on, this may be a sign that the user is about        to capture images of a new scene. Based on this, the camera may        ask the user a question (e.g., “Are you indoors or outdoors?”).    -   (ii) A user captures an image. For example, the camera may        output a question to a user immediately after the user captures        an image of the scene (e.g., in the anticipation that the user        will capture additional images of the scene).    -   (iii) A user presses the shutter button. For example, the camera        may output a question to a user when the user presses the        shutter button on the camera halfway (e.g., to focus the        camera's lens on a subject). This may indicate that the user is        about to capture an image of the subject.    -   (iv) A user moves to a new location. For example, the camera may        include a GPS sensor or other location sensor that allows it to        determine when a user moves the camera to a new location. Since        this may be a sign that the user is now capturing images of a        new scene, a question may be output to the user (e.g., “Are you        still taking pictures of Alice?”).    -   (v) A user operates a control. For example, the camera may        output a question to a user when the user adjusts a setting on        the camera, since this may be an indication that some aspect of        the camera's settings need to be adjusted (e.g., because the        user is capturing images of a different subject).

The camera may store an output condition database such as the one shownin FIG. 11. Note that the output condition database shown in FIG. 11specifies how one or more questions should be output to a user. Forexample, if the camera is currently in manual mode, the camera willoutput a question to a user by beeping and displaying the question astext along the bottom of the camera's viewfinder. Note that thisexemplary version of the output condition database may be used by thecamera to output a question based on a current mode of the camera andpossibly one or more other factors. For example, according to the outputcondition database shown in FIG. 11, the camera is currently in “Manual”mode, so questions may be output to a user when the user looks throughthe camera's viewfinder. An alternate version of the output conditiondatabase might be used to output a question based on other factors.

There may be times when a user would prefer not to have a questionoutput to him (e.g., when he is busy with another activity, or whenoutputting a question would be disturbing to the user or other people).In circumstances like these, output of a question may be suppressed(e.g., by the camera) based on one or more conditions. Suppressing aquestion may include, without limitation: preventing a question frombeing output, not outputting a question, canceling output of a question,and delaying output of the question.

It will be readily understood that a question may be determined, asdiscussed variously understood, but that its output is suppressed,delayed, or cancelled. Also, suppression of a question does notnecessarily mean that no questions are output at all. Suppression mayinclude suppressing one or more questions (or types of questions) whileone or more other questions are output. For example, two questions maybe identified at about the same time for output to a user, with onebeing output immediately and output of the second question being delayeduntil a more appropriate time.

Conditions for suppressing questions may be similar to those describedabove for outputting questions. For example, a question may besuppressed based on an indication from a user or because a time limitexpires.

In some embodiments, a question may be suppressed because it comes at aninappropriate time. For example, it may be determined that a user maycurrently be busy with another activity. In another example, if a useris busy adjusting the camera's settings, then the camera may delayoutputting a question until the user finishes adjusting the camera'ssettings and captures the image that he was busing composing.

According to some embodiments, one or more questions may be suppressedbecause of inappropriate content. For example, a determined question maybe a duplicate of a recently-asked question. Since the user has alreadyanswered the question, it may be bothersome to ask the user the samequestion again. In another example, the answer to (or purpose of) aquestion may no longer be relevant. For instance, while a user isindoors, the camera may determine a question to ask the user: “What typeof light bulb does this room have, tungsten or fluorescent?” However,the camera may have delayed outputting this question, because the useris in the middle of a conversation with a friend, for example. If theuser then moves outside before the question is output, that question isno longer relevant.

A user may indicate that one or more messages (or types of messages)should be suppressed. For example, a user who is capturing images at agolf tournament may indicate that no audio message should be output(e.g., so that the user does not disturb the golfers).

Suppressing a question may include removing the question from an outputqueue or other list of questions to be output to a user.

The output condition database shown in FIG. 11 shows an example ofdelaying output of a question if the camera is in “Sports” mode In thisexemplary mode, the camera may delay outputting a question to a useruntil the camera is held still for a period of time. Another example ofFIG. 11 relates to canceling output of a question. For example, thecamera may refrain from outputting any questions when it is in “Do NotDisturb” mode.

According to some embodiments of the present invention, a camera mayoutput an indication of a question before outputting the questionitself. For instance, rather than outputting a determined questionimmediately, the camera may output an indication that the question isready to be output. In some embodiments, the camera will then wait for auser to indicate that he is ready for the question itself to be output.For example, the camera may beep when it determines a question and thenwait for the user to respond to this beep before outputting thequestion. In a second example, an LED on the camera may flash wheneverthe camera has a question ready to output to a user.

According to some embodiments, outputting a question to a user mayinclude one or more of the following steps: outputting an indicationthat a question is ready, receiving a response to the indication fromthe user, and outputting the question based on the response. Examples ofeach of these steps are provided below.

Some examples of outputting an indication that a question is readyinclude, without limitation: outputting an indication that a questionhas been determined, outputting an indication that a question has beenqueued for output, outputting an indication that a question has beenretrieved from memory, outputting an indication that a question is readyto be output, outputting a request for the user's attention, andoutputting a request for the user's attention regarding a question. Someof these examples are discussed in more detail herein.

It will be readily understood that there are many ways of indicating toa user that a question is ready, such as by use of the various outputdevices discussed herein. For example, outputting an indication maycomprise illuminating an LED on a camera. A user may understand thatwhenever this LED is illuminated, the camera has a question queued toask the user. In another example, an audible “BEEP” or bell sound may beoutput that a user may hear. A user may understand that whenever thisbeep sounds, the camera has a question queued to ask the user. A messagemay be displayed on an LCD screen. For example, an LCD screen on theback of the camera may display a text, “I've got a question. Press the‘Ask Me a Question’ button to have this question output to you.”

In still another example, a portion of an image may be highlighted(e.g., in the camera's viewfinder). For instance, if the camera has aquestion about a particular subject in an image, the camera mayhighlight that subject in red when the image is displayed on thecamera's LCD viewfinder. Different types of visual indicators may beused to alert a user that at least one question is pending. In oneexample, when a user views an image using the camera, the camera mayplace a green border around the image to indicate that there is aquestion associated with the image. In another example, a red questionmark may be overlaid on the corner of a displayed image to indicate thatthe camera has a question related to the image.

It will be readily understood that various methods described herein foroutputting a question may also be used to output an indication that aquestion is ready. For example, an indication that a question is readymay include a presenter (e.g., an animated character, a celebrityvoice).

The camera may output an indication that a question is ready based onone or more conditions. Note that the various output conditionsdescribed herein for outputting a question may also be used to controlthe output of an indication that a question is ready.

According to some embodiments, a user may respond at his leisure to anindication that a question is ready. For example, at 2:12 p.m. thecamera may beep to indicate that it has a question to ask a user. Theuser may be busy with some other activity at this time (e.g., capturingan important sequence of images of a sporting event) and so ignores thisbeep until 2:17 p.m., when he is free to pay attention to the questionthat the camera outputs. To indicate that he would like to answer aquestion, the user may operate a control on the camera (e.g., press abutton).

Receiving a response to an indication that a question is ready mayinclude, without limitation: receiving an indication from a user,receiving an indication that a user would like to view a question, andreceiving an indication that a user is inactive. A user may provide aresponse by operating a control or other input device on the camera.Various types of input devices are discussed herein, and others may bereadily apparent to one skilled in the art in light of the presentdisclosure.

A user's response may include an indication of which question should beoutput. For example, a user may indicate that he is ready to answerquestions about lighting, scenes, and future plans, but not aboutmeta-tagging. In a second example, a user may select a question toanswer from a list of questions that have been determined.

A user's response may include an indication of how to output a question.For example, a user may prefer to have a question output to him withboth audio (e.g., through a speaker on the camera) and text (e.g.,through a LCD display on the camera).

Some types of users may prefer to have the camera output an indicationthat a question is ready before outputting the question. Other users mayprefer to have the camera determine the best time(s) to output aquestion. For example, allowing the user to control when and how aquestion is output may be preferred by users because of the control andsimplicity such a system offers. For instance, a user may wish to havecontrol over the outputting of questions that he finds annoying.

The output condition database in FIG. 11 shows a number of examples ofhow the camera may output an indication that a question is ready beforeoutputting the question itself. According to one example, when thecamera is in “Output Upon Request” mode, the camera will beep when aquestion is ready to be output and then wait until a user presses thecamera's “Ask Me a Question” button before outputting a question. Inanother example, when the camera is in “Silent” mode, the camera willcause an LED to blink to indicate that a question is ready.

According to various embodiments of the present invention, a user mayindicate a response to a question that is output to him. For example,the camera may output a question to a user, “Are we on a beach?” and theuser may reply “No.”

A user may operate one or more controls or other input devices of thecamera to indicate a response to a question. Various types of inputdevices and controls that the camera may include are described herein,and other types may be readily apparent to those skilled in the art inlight of the present disclosure. For example, a user may use one or morebuttons on the camera to select a response from a plurality of response(e.g., an answer to a multiple-choice question). For instance, a usermay select a response from the list of choices: “sunny,” “partlycloudy,” “light rain,” “heavy rain,” “light snow,” “heavy snow.” Inanother example, a user may press a button on the camera to indicate“Yes” in response to a question of whether he is capturing images of asporting event. In another example, the camera may include a microphonethat allows a user to respond to a question verbally (e.g., a user mayindicate the weather outside is sunny by saying the word “Sunny”). Insome embodiments, a user may use a stylus or other device to spell out aresponse on a touch screen on the camera. For example, a user may usethe Graffiti™ alphabet to spell out a textual response to a question.

Some embodiments of the present invention allow for a user to speak aresponse to a question. Such a response may be recorded using amicrophone on the camera. The camera may then process the response usingvoice recognition software. For example, a user may indicate that he isat a “birthday party” and the weather is “raining.”

In another example, a user may use any of a plurality of input devices(e.g., buttons) on the camera to highlight a portion of an imagedisplayed (e.g., on the camera's color LCD display). For example, thecamera may ask a user to indicate where a subject's face is in an image.Based on the user's response, the camera and/or a server may determinethat this area of the image is properly exposed.

It will be readily understood that a user's response to a question maytake a variety of different forms (e.g., depending on the type ofquestion). Some examples of forms of answers include, withoutlimitation:

-   -   (i) Yes/No (e.g., an answer to the question “Is this a picture        of Alice?”).    -   (ii) Open-ended. For example, a user may respond to the question        by speaking freely (e.g., by speaking “Alice” in response to a        question about who is in a displayed image).    -   (iii) A selection from a plurality of choices. For example, a        user may respond to the question, “Who is in this picture?” by        selecting one of the offered choices: “a) Alice b) Bob c)        both d) neither.”    -   (iv) Graphical response. For example, a user may highlight a        portion of an image or point to an object in an image displayed        on a LCD display on a camera.

The camera may also receive or otherwise determine a default responsefrom a user. For example, if a user does not respond to a question in acertain period of time, the camera may assume that the user answered thequestion in a certain way (e.g., in accordance with a default answer, asdiscussed herein). For example, the camera may ask a user the question“You're at a ski resort, aren't you?” If the user does not respond tothis question within ten seconds, then the camera may assume that theuser answered “Yes” to this question.

According to some embodiments, a camera and/or server may verify aresponse from a user. Verifying a response may include outputting anindication of the response that the user provided, or asking the samequestion or a similar question. The camera may verify a response toensure that it did not misunderstand the user's response. For example, auser may respond “Yeah” or “Nah” to a Yes/No question, making itdifficult for the camera to use voice recognition software to determinethe user's response. In order to verify the user's response, the cameramay output an indication of the user's response by displaying thecamera's best guess as to the response (e.g., “Yes”) on the camera's LCDdisplay. In some embodiments, the camera may verify a response to ensurethat a user did not make a mistake in responding to a question. Forexample, the camera may output a second question to verify that a userdid not accidentally press the wrong button on the camera whenindicating his response (e.g., “Are you sure that this room hasfluorescent light bulbs?”).

The camera may verify a response to confirm that the user understandsthe ramifications of his response. For example, a user may indicate thathe only plans to capture five more images on the camera's current memorycard. Based on this, the camera may output a warning or reminder: “Areyou sure that you only want to capture 5 more images? If you capture 5more images at high resolution, then the camera will be out of memoryand will not be able to store any more images.” According to someembodiments, the camera may verify a response by displaying an image toa user. For example, a user may respond to a question by indicating thathe is at a beach. Based on this, the camera may display an image to auser along with the message, “This is what the picture would look likein Beach Mode. Is this correct?” The user may then respond to theverification by indicating whether the displayed image is correct.

In some cases a user may not respond to a question. For example, a usermay ignore a question that is output by the camera. For instance, thecamera may output a question when a user is busy with another activity(e.g., capturing an important sequence of action shots at a sportingevent). Instead of responding to the question, the user may ignore thequestion. In another example, a user may not know that the camera outputa question. For instance, a user may not be looking at the camera's LCDscreen when a question is displayed on the LCD screen. Since the userdoes not know that the camera has output a question, he will not know torespond to the question.

Determining that a user has not responded to a question may includedetermining that a period of time has elapsed since the question wasoutput. For example, if a user does not respond within twenty seconds ofwhen a question is output, then the camera may determine that the userhas not responded. The camera may then perform an action (e.g., outputthe question again). In some embodiments, determining that a user hasnot responded to a question may comprise determining that a conditionhas occurred (e.g., a user presses a button on the camera, the camera isheld motionless for a period of time, a user provides a response to adifferent question). For example, the camera may stop displaying aquestion when a user captures an image.

If a user does not respond to a question output by the camera, then thecamera take one or more of the following actions:

-   -   (i) Continue outputting the question. For example, the camera        may display a question on an LCD screen until the user responds        to the question.    -   (ii) Stop outputting the question. For example, the camera may        display a question on an LCD screen until either (a) a user        responds to the question, or (b) thirty seconds have elapsed.    -   (iii) Output the question again. For example, the camera may        output an audio recording of a question. If a user does not        respond, then the camera may output the question again (perhaps        in a different form).    -   (iv) Assume a default response to the question. For example, the        camera may output a question to a user, “You're taking a picture        of a sunset, aren't you?” If the user does not respond, the        camera may assume that the answer to the question is “Yes”        (e.g., based on a determined default answer) and perform an        appropriate action based on this default response.    -   (v) Output a different question. For example, the camera may ask        a user a first question, “Are you at the beach?” If the user        does not respond to the question, then the camera may ask the        user a second question (e.g., based on the same image), “Are you        at a ski resort?”

It will be understood that some times a user may provide a response thatdoes not answer a question. For example, a user may indicate that hedoes not want to answer a question. Accordingly, a user may press a“Cancel Question” button on the camera. In a second example, a user mayuse a control on the camera to put the camera in “Do Not Disturb” mode,thereby canceling the current question and any future questions.According to some embodiments, a user may indicate that he would preferto answer a question at a later time. For example, the camera may have a“snooze button” that allows a user to indicate that the camera shouldstop outputting a question and then output the question again when acondition occurs (e.g., a period of time has elapsed, an outputcondition occurs).

In at least one embodiment, a user may indicate whether a question isinappropriate or unhelpful. For example, the camera may have a “thumbsdown” button or a “stupid question” button that a user may press whenthe camera outputs a question that the user determines is not worthanswering. This may be particularly useful if the camera tends to makemistakes when determining questions to output to users. For example, auser may capture a plurality of images indoors and then move outdoors tocapture more images. Based on the plurality of images captured indoors,the camera may ask the user, “What kind of lighting does this roomhave?” Since the user is currently outdoors, this question isinappropriate, so the user may press the “thumbs down” button toindicate that the camera should discard the question about lighting asbeing irrelevant to the current situation. Similarly, the camera mayhave a “reset questions” button that allows a user to indicate that thecamera should restart its line of questioning.

In accordance with one or more embodiments of the present invention, ifa user does not respond to a question or a user provides a response thatdoes not answer the question, then the camera may output the questionagain. The second output of the question may be similar to or differentfrom the first output of the question. A question may be output a secondtime based on a different output condition. For example, a question maybe output when a user presses the shutter button halfway down (an outputcondition). If a user does not respond to this output of the question,then the camera may output the question again in fifteen seconds (asecond output condition).

In some embodiments, a question may be output a second time based on thesame output condition. For example, a question may be output thirtyseconds after a user operates the camera (an output condition). If theuser does not respond to this first output of the question, then thecamera may output the question a second time in response to a secondoccurrence of the output condition. That is, the camera may wait untilthe user operates the camera again, and then output the question thirtyseconds after the user stops operating the camera.

Of course, a question may be output in the same manner as it waspreviously output. For example, a question may be output a first time asan audio prompt. If a user does not respond to the question, then thecamera may repeat the audio prompt. In another embodiment, a questionmay be output in a manner that is different from the one that waspreviously used for the question. For example, a question may be outputa first time as text displayed on the camera's LCD screen. If a userdoes not respond, then the camera may output the question as holographictext overlaid on the camera's optical viewfinder.

Some embodiments allow for a question to be output with or withoutadditional information. For example, a question may be output a firsttime as “Are you taking a picture of a sunset?” If a user does notrespond to this first output of the question, then the camera may outputthe question again, this time providing additional information: “Are youtaking a picture of a sunset? If so, then let me know now—otherwise yourpicture will be underexposed.”

The camera may store an indication of a user's response to a question.For example, the camera may store an indication of a response in aresponse database such as the one shown in FIGS. 12A and 12B.

Storing an indication of a user's response to a question may be helpfulin a variety of different circumstances, including some exemplaryscenarios described herein. Other uses of one or more stored indicationsof a user's response(s) will be readily apparent to those skilled in theart in light of the present disclosure.

A stored indication of a response may be useful in performing an actionbased on multiple responses. For example, the camera may ask a user aplurality of questions and receive a plurality of responses from theuser. The camera may then perform an action (e.g., adjust a setting,meta-tag an image, guide a user in operating the camera) based on theplurality of responses. For example, a user may indicate in a firstresponse that he is capturing a picture of at a ski resort, and thisfirst response may be stored in a database (such as the responsedatabase). Later, in a second response, the user may indicate that theweather is cloudy. Based on these two responses, the camera may adjustthe settings on the camera to appropriate values for capturing images ata ski resort during cloudy weather.

Indications of responses may be beneficial in determining futurequestions to ask a user. For example, the camera may ask a user a firstquestion (e.g., “Are you indoors or outdoors?”). The user may thenrespond to this question (e.g., “Indoors”) and the camera may store thisresponse. Based on the stored response, the camera may ask the user asecond question (e.g., “What kind of lightbulbs does this room have?”).

Storing an indication of a user's response may assist a computing device(e.g., a camera, a server) in avoiding repeating questions or askingunnecessary questions. For example, the camera may avoid asking a userthe same question twice in close succession by checking to see if theuser has already answered the question recently. If the user hasanswered the question recently, then the camera may assume that theuser's answer is unchanged. For example, the camera may ask user if heis on a beach and store the user's response (e.g., “Yes”). Ten minuteslater, the camera may refrain from again asking the user if he is on abeach and instead assume that the answer from ten minutes before isstill valid and that the user is still on the beach. In at least oneembodiment, as discussed further herein, information provided by a usermay expire after a certain period of time or based on some othercondition. For example, a camera may store an indication of a user'sresponse in an expiring information database, such as the one depictedin FIG. 14.

A camera and/or a server may perform various actions based on a responsefrom a user, including one or more of: meta-tagging an image, adjustinga setting, guiding a user in operating a camera, outputting a secondquestion, and determining a template.

A computer server may assist the camera in processing a user's responseto a question. Examples include, without limitation:

-   -   (i) The camera may receive a user's response to a question and        transmit an indication of this response to the computer server.        The computer server may then process this response (e.g., using        voice recognition software) to determine an appropriate action        based on the response (e.g., adjusting a setting on the camera).        An indication of the action may then be transmitted to the        camera and performed by the camera.    -   (ii) The computer server may transmit instructions to the camera        describing how to process a response by a user. For example, in        addition to transmitting an indication of a question to the        camera (as described above), the computer server may transmit        one or more sets of instructions describing how to process a        user's potential responses to the question. For example, the        computer server may indicate that if the user responds “Yes” to        a question, then the camera should put the flash in slow-syncro        mode; if the user responds “No” to the question, then the camera        should ask the user whether he is outdoors. Note that        instructions may be transmitted to a camera in a variety of        different forms, including a computer program (e.g., in C or        Java), script (e.g., in Javascript or VSscript), or machine code        (e.g., x86 assembly).

Meta-tagging may be used herein to refer generally to a process ofassociating supplementary information with an image (e.g., that iscaptured by a camera or by some other imaging device). The supplementaryinformation associated with an image may be referred to as meta-data,meta-information, or a meta-tag. Some examples of meta-data include,without limitation:

-   -   (i) A time and date when an image was captured (e.g., Oct. 10,        2002)    -   (ii) A location where an image was captured (e.g., latitude and        longitude coordinates obtained from a GPS sensor, an indication        of a city, state, park, or other region provided by a user, an        altitude determined using an altimeter). For example, a user may        indicate that an image was captured in the SoHo area of New York        City.    -   (iii) An orientation of the camera when an image was captured        (e.g., determined using tilt sensor or an electronic compass)    -   (iv) One or more subjects of an image (e.g., people, objects,        locations, animals, etc.). Note that a subject may be uniquely        identified (e.g., “Alice Jones,” “Grand Canyon”) and/or        categorized (e.g., “a squirrel,” “national park”).    -   (v) A scene in an image (e.g., a rainbow next to a waterfall, a        group of friends at a restaurant, a baby and a dog, a family        portrait, a reflection in a mirror)    -   (vi) Motion relating to an image (e.g., movement of a subject,        movement of the camera)    -   (vii) The environment in which an image was captured (e.g.,        weather conditions, sunlight, altitude, temperature)    -   (viii) Lighting (e.g., daylight/a tungsten light bulb, a        florescent light bulb, flash intensity, locations of light        sources)    -   (ix) One or more settings on the camera (e.g., aperture, shutter        speed, flash, mode). For example, meta-data associated with an        image may indicate that the image was captured at f/2.8 with a        CCD sensitivity of 100 ISO and a shutter speed of 1/250 sec.    -   (x) Information about how an image was captured. For example, an        image may be meta-tagged as being part of an auto-bracketed        group.    -   (xi) Information about a user (e.g., the user's name,        preferences, priorities)    -   (xii) Preferred cropping or scale. For example, meta-data        associated with an image may indicate what portion of the image        should be printed and/or how large the image should be printed.    -   (xiii) A category for an image. For example, an image may be        categorized based on its intended usage (e.g., part of a slide        show), based on its subject (e.g., images of Alice), and/or        based on how or when it was captured (e.g., captured during a        ski trip on Dec. 7, 2002).    -   (xiv) A user's intentions (or other notes from a user). For        example, a user may indicate, “I'm trying to get a picture of        the baby with its eyes open” or, “I want to capture the        reflection of the mountain in the water of the lake” or,        “Getting the exposure right for the subject's face is most        important; I don't care whether the background is in focus.”    -   (xv) An audio clip. For example, the camera may record a ten        second audio clip when capturing an image and stored this audio        clip as meta-data associated with the image.    -   (xvi) Acceptable to delete. For example, in his response to        question a user may indicate to the camera that it is acceptable        to delete one or more images from the camera's memory in order        to make room for images that may be captured in the future.        Based on this response, the camera may meta-tag one or more        images for “deletion,” meaning that these images may be deleted        if the camera begins to run out of memory.    -   (xvii) Protection. For example, the camera may meta-tag one or        more images as being “protected,” meaning that these images        should not be deleted or altered in any way. The “protected”        meta-tag may be helpful in ensuring that the user or the camera        does not inadvertently delete one of the protected images.    -   (xviii) A rating. For example, the camera may determine a rating        of an image and store this rating with the image. A rating may        be an indication of the quality of the image and may be based        one a variety of different factors, including: exposure,        sharpness, composition, subject, and indications from a user.        Ratings may be helpful in allowing the camera to sort images.

Note that meta-data that is associated with an image may be determinedbased on one or more responses indicated by a user. For example, a usermay indicate in a first response that he is in Maui. In a secondresponse, the user may indicate that he is at the beach. Based on thesetwo responses, the camera may meta-tag an image as being taken “On thebeach in Maui.”

According to various embodiments of the present invention, a camera maymeta-tag an image based on a user's response to a question. For example,a server may transmit to a camera a signal indicating that a recordedimage is most likely of Alice (e.g., based on an image recognitionprogram). The camera may then ask a user to verify that the image is ofAlice. If the user indicates that the image is of Alice, then the cameramay meta-tag the image as “Subject of Image: Alice.” In another example,if a user indicates that an image shows “Alice and Bob in Yosemite,”then the camera may meta-tag the image as “Subjects: Alice,Bob//Location: Yosemite National Park.”

In another example, an image database may be used by a server 110 inperforming an image recognition process on a captured image. Accordingto some embodiments of the present invention, if the recognition processmatches a stored image (e.g., “YOSEMITE-08”) to a new image, the server110 may suggest some or all of the meta-data 830 associated with thestored image to a user (e.g., by transmitting an indication of themeta-data 830 to the camera 130). The user may then conveniently agree(e.g., by pressing an “Ok” button) to have the suggested meta-dataassociated with the new image. In this manner, a user may avoid some ofthe tedium of creating meta-tags.

Further, any new images may be stored in the image database and thus maybe made available to an image recognition process. In this way, an imagerecognition process and/or a process for meta-tagging images may berefined or customized in accordance with the stored meta-informationassociated with a particular user's images. For example, a first usermay have captured an image of a particular scene, associated meta-dataincluding the description “Grand Canyon” with the image, and stored theimage on his personal computer. A second user may have captured andstored a very similar (or identical) image, but associated with theimage (e.g., as meta-data 830) the description, “Arizona, Grand Canyon,March 1999.” If the first user transmits a second image similar to thestored image to his personal computer (e.g., from the camera 130 viacommunications network 120) for image recognition, the computer mayidentify the same scene or subject based on the stored image, andsuggest “Grand Canyon” to the first user. The second user's server 110,however, might suggest, for example, one or more of “Arizona,” “GrandCanyon,” and “March 1999” for the same second image. Thus, an imagedatabase may be useful in accordance with some embodiments of thepresent invention for generating and/or suggesting personalizedmeta-information for a particular user (or group of users).

A plurality of images may be meta-tagged based on at least one responsefrom a user. For example, a user may indicate that he is at the beach.Base on this, all images captured by the camera may be meta-tagged asbeing captured “At the Beach.” In another example, a user may indicatein a response to a question that Alice is the only blonde woman who hehas captured any images of today. Based on this, the camera mayautomatically meta-tag all images of blonde women taken today as beingimages of Alice.

It will be readily understood by those skilled in the art that a varietyof different forms of meta-data are possible, including, withoutlimitation: text (e.g., a current date, a GPS location, the name of asubject, a current lighting condition), audio (e.g., an audio recordingof a user's response to a question), images (e.g., a user's response maybe a highlighted portion of an image), binary or other machine-readableformats (e.g., a 100 bytes of information at the start of an imagefile), and any combination thereof.

Meta-data may also be stored in a variety of different ways. Forexample, meta-data may be stored in a file that is separate from animage file to which it pertains. For example, a “BOB23.TXT” file maystore meta-data that pertains to a “BOB23.JPG” image that is stored bythe camera. In some embodiments, meta-data may be stored in an imagefile. For example, the start of an image file may include a plurality ofmeta-tags that provide information based on a user's responses to one ormore questions. In one or more embodiments, a single meta-data file maystore information for a plurality of images. For example, the camera maystore a response database that includes meta-data for a plurality ofimages (see below for further details). According to some embodiments ofthe present invention, a camera may store an audio clip of the user'sresponse to a question and associate this audio clip with an image asmeta-data. In another exemplary embodiment, a camera may set the filename of an image based on a user's response to a question. For instance,if a user indicates that an images is of Alice, then the camera maystore this image with the filename “ALICE-01.JPG.”

A wide variety of methods of meta-tagging an image and of differenttypes of meta-data are known to those skilled in the art and need not bedescribed in further detail herein. For additional reference, the readermay refer to U.S. Pat. No. 6,408,301 to Patton, et al., entitled“Interactive image storage, indexing and retrieval system”; U.S. Pat.No. 5,633,678 to Parulski et al., entitled “Electronic still camera forcapturing and categorizing images”; and U.S. Patent Application No.20010012062 by Anderson et al., entitled “System and method forautomatic analysis and categorization of images in an electronic imagingdevice.”

The response database shown in FIG. 7 and the image database shown inFIG. 8 depict a few examples of a camera meta-tagging an image based ona user's response to a question. For instance, the image “WEDDING-02”was meta-tagged as including “Alice” and “Bob” (e.g., based on a user'sresponse to question “QUES-123478-03”). The exemplary images“BEACHTRIP-05” and “BEACHTRIP-06” were meta-tagged as images of a beach(e.g., based on a user's response to “QUES-123478-06”).

Referring to FIG. 17, a flowchart illustrates a process 1700 that isconsistent with one or more embodiments of the present invention. Theprocess 1700 is a method for determining a question based on informationdetermined based on an image recognition process performed by a server.For illustrative purposes only, the process 1700 is described as beingperformed by a camera 130 in communication with a server 110. Of course,the process 1700 may be performed by any type of imaging device 210 incommunication with a computing 220.

In step 1705, the camera 130 captures an image. In step 1710, the camera130 transmits the image to the server 110 for image recognitionprocessing. For example, as discussed herein, the server 110 may comparethe captured image to a database of images stored for the user in adatabase. In step 1715 the camera receives information determined by theserver 110 based on the image recognition process. For example, theserver 110 may have matched the captured image to a stored image,retrieved the meta-information associated with the stored image, andforwarded the meta-information to the camera 130. In another example,the server 110 may have been unable to identify a match and may havetransmitted a signal to the camera 130 directing the camera 130 to askthe user if the user would like to apply the same camera settings in thefuture to any similar images.

In step 1720, the camera 130 determines a question based on theinformation from the server. For example, the camera may generate aquestion asking if the user would like to associate meta-informationreceived from the server 110 with the newly-captured image. The questionis output to the user in step 1725. In step 1730 the camera 130 receivesa response from the user and the camera performs an action based on theresponse (step 1735). For example, the camera may associate meta-datawith the captured image based on the response.

Referring to FIG. 18, a flowchart illustrates a process 1800 that isconsistent with one or more embodiments of the present invention. Theprocess 1800 is a method for determining meta-information. Forillustrative purposes only, the process 1800 is described as beingperformed by a server 110 in communication with a camera 130. Of course,the process 1800 may be performed by any type of imaging device 210 incommunication with a computing 220.

In step 1805, the server 110 receives an image captured by a user of acamera 130. In step 1810, the server 110 determines at least one of aplurality of images meta-tagged by the user. For example, the server 110may access the user's personal images database that contains imagespreviously meta-tagged by the user. In step 1815, the server 110determines meta-information to suggest to the user based on the capturedimage and the at least one image meta-tagged by the user. For example,using an image recognition program, the server 110 identifies one ormore matches for the captured image in the user's database of images andretrieves some or all of the meta-information associated with thosematching images.

In step 1820, the server 110 transmits an indication of themeta-information to be suggested to the user. In an optional step 1830,the server 110 receives an indication from the camera 130 ofmeta-information associated with the captured image by the user. Forexample, the camera 130 may transmit a signal indicating that the userhas accepted the suggested meta-information, or, alternatively, maytransmit a signal indicating other meta-information the user has decidedto associate with the image (e.g., the user may have rejected all orsome of the suggested information and may have provided othersupplemental information).

According to various embodiments of the present invention, a camera mayautomatically adjust one or more of its settings based on a responsefrom a user and/or based on a signal from a server. For example, if auser indicates in his response to a question that the weather is sunny,then the camera may adjust the aperture on the camera to be f/5.6 andthe shutter speed to be 1/250 sec. Various different types of settingson the camera are described in detail herein, and others will be readilyunderstood by those skilled in the art. Some examples of settings on thecamera include, without limitation: exposure settings, lens settings,digitization settings, flash settings, multi-frame settings, powersettings, output settings, function settings, and modes. Adjusting asetting may include, without limitation, one or more of: turning asetting on or off, increasing the value of a setting, decreasing thevalue of a setting, modifying a setting, changing a setting, revising asetting, and setting the camera to capture an image in a particularmanner.

The following are only some examples of how one or more settings may beadjusted based on a user's response to a question in accordance with oneor more embodiments of the present invention. Each exemplary scenariocomprises an exemplary question asked by a camera, a response from auser, and action(s) performed by the camera:

-   -   (i) Question asked by camera: “Is this photo too dark?”        -   Response from user: “Yes”        -   Action: Increase the aperture on the camera.    -   (ii) Question asked by camera: “Are you taking pictures of a        sporting event?”        -   Response from user: “Yes”        -   Action: Put camera in burst mode to take 3 pictures every            time the shutter button is pressed and increase the shutter            speed to that it is faster than 1/250 sec.    -   (iii) Question asked by camera: “What are the current weather        conditions?”        -   Response from user: “Cloudy and Overcast”        -   Action: Adjust white balance (color temperature) on camera            to 6000 K.    -   (iv) Question asked by camera: “How far away is the wall behind        your subject?”        -   Response from user: “About 10 ft”        -   Action: Adjust flash timing and shutter speed for slow            synchro.    -   (v) Question asked by camera: “Which is more important to you,        taking more pictures or getting higher resolution pictures?”        -   Response from user: “Taking more pictures”        -   Action: Set JPG compression to “medium quality” and            resolution to “regular.”    -   (vi) Question asked by camera: “Do you want the building behind        your subject to be in focus?”        -   Response from user: “Yes”        -   Action: Set aperture on camera to f/8 (or less). Adjust            shutter speed or film speed accordingly.    -   (vii) Question asked by camera: “Are we at a ski resort?”        -   Response from user: “Yes”        -   Action: Adjust white balance and exposure metering for            subjects on bright white backgrounds.    -   (viii) Question asked by camera: “Are you taking a group photo?”        -   Response from user: “Yes”        -   Action: Crop the image to include everybody in the group.    -   (ix) Question asked by camera: “Are you on a boat?”        -   Response from user: “Yes”        -   Action: Adjust image stabilization setting to “high.”

According to some embodiments, a camera may adjust a setting for one ormore images. For example, the camera may output a question to a user,“Is this a group photo?” If the user responds “Yes” to the question,then the camera may adjust one or more settings and enable the user tocapture one image based on these settings. After capturing the image ofthe group of people, the camera may revert to its original settings, forexample, or determine one or more new settings for capturing images inthe future. In some embodiments, settings may be adjusted for aplurality of images. For example, the camera may output a question to auser, “Are we at the beach?” If the user responds “Yes” to the question,then the camera may put the camera in “Beach” mode for the remainder ofthe user's image-capturing session. The camera may remain in “Beach”mode until the user turns the camera off or until the user beginscapturing images of a different scene.

An adjustment to a setting may persist until a condition occurs. Variousexamples of conditions are described herein, and others may be readilyapparent to one skilled in the art in light of the present disclosure.

A camera may not immediately adjust a setting based on a user's responseto a question. For example, a camera may ask the user a plurality ofquestions and then adjust at least one setting based on the user'sresponses to the plurality of questions. In a second example, the cameramay not adjust a setting based on a first question until after a userhas answered a second, related question.

According to some embodiments, the camera may indicate to a user anadjustment made to a setting. For example, a user may respond to aquestion by indicating that he is at the beach. Based on this, thecamera may increase the color saturation setting on the camera by 5%. Inaddition, the camera may output a message to the user “Increasing colorsaturation 5%.”

Indicating an adjustment to a setting may be helpful for a variety ofreasons, such as by assuring the user that the camera is in fact makinguse of his responses to questions. For example, even if a user does notunderstand what adjustment the camera is making, he may find itcomforting to be informed that the camera is making use of his responsesto questions. If a user were to feel that his responses to questionswere being ignored by the camera, then he might ignore future questionsthat are output by the camera.

Indication of an adjustment may be helpful to the user in verifying thatthe camera has not misunderstood or misinterpreted a user's response toa question. For example, a user may respond to a question by indicatingthat he is at a football game. Based on this indication and the currenttime of day (e.g., 2 p.m.) the camera may assume that the game is beingilluminated by sunlight. However, in fact, this may not be the case(e.g., the football game may be played in a domed stadium). When thecamera indicates that it is “Adjusting the camera for sports duringdaylight conditions,” the user may notice this mistake and correct thecamera by indicating that the football game is in fact illuminated byhalogen light bulbs. Informing the user of any adjustments that are madeto the camera may also help the user in composing a shot or in makingfurther adjustments to the camera's settings. For example, informing theuser that the “flash brightness has been set for subjects 10-12 feetaway” may be helpful to a user if the user decides to move or recomposean image.

Instead of automatically making an adjustment, a camera may ask for auser's permission before making an adjustment to a setting. If the userindicates that it is acceptable to adjust the setting on the camera,then the camera may adjust the setting. If the user indicates that hewould rather not adjust the setting on the camera, then the camera maynot adjust the setting. For example, based on a user's response, thecamera may determine that the camera's flash should be turned on. Beforeturning on the flash, the camera may output a message to the user, “I'mabout to turn on the flash. Is this okay?” If the user responds “Yes,”then the camera may turn the flash on. In another example, the cameramay determine that a user is capturing an image of a sunset based on theuser's responses to one or more questions. Based on this, the camera mayoutput a question to the user, “Would you like to put the camera intoSunset Mode? Sunset Mode is specially designed to make sure thatpictures of sunsets are exposed correctly.” The user may then press a“Yes” button on the camera to indicate that he would like to put thecamera into “Sunset” mode.

Asking for a user's permission to adjust a setting on the camera may besimilar to providing advice to a user about adjusting a setting. Variousways of providing advice to a user based on the user's response to aquestion are discussed herein.

The camera may implement one or more rules based on a user's response toa question. A rule may be a guideline or other indication that may beused to determine a setting on the camera. Implementing a rule mayinclude one or more of: storing an indication of a rule in memory,automatically adjusting a setting of the camera based on a rule, andrestricting operation of the camera based on a rule.

In one example of implementing a rule, a user may respond to a questionby indicating that he is capturing images of a child's birthday party.Based on this, the camera may store a rule that requires that the cameramaintain shutter speed of at least 1/125 sec (because children at abirthday party tend to move quickly), except when the camera determinesthat an image includes a birthday cake with candles, in which case thecamera should set the aperture to be a large as possible and not use aflash. An indication of a rule may be stored, for example, in a rulesdatabase (not shown) or a settings database such as the one depicted inFIG. 7.

In another example of use of a rule, a rule may be a requiredrelationship between one or more settings. For example, based on auser's indication that he is taking pictures at the beach, the cameramay ensure that the subject of an image is always correctly exposed,even if the background of the image is overexposed. In a relatedexample, the camera may use an automatic neutral density feature toautomatically vary the exposure of the subject relative to the exposureof the background. In another example, a user may respond to a questionby indicating that he is capturing images at a zoo. Based on this, thecamera may implement a rule that, if the user is outdoors, the camera'saperture should be smaller than f/8 (to ensure good depth of field). Ifthe user is indoors, a rule may establish that the camera shouldincrease the CCD sensitivity as much as possible and never use a flash(to avoid frightening the animals).

According to some embodiments, a rule may indicate how a setting on thecamera should be adjusted. For example, based on an indication from auser that Alice is standing in front of a tree, the camera may implementa rule to shift the hue of an image by +5% anytime the camera is used tocapture an image of Alice wearing her green jacket (e.g., to avoidhaving Alice's green jacket blend into the background. In anotherexample, a user may respond to a question by indicating that he is at aski resort. Based on this, the camera may implement a rule that until 5p.m. that day, all images captured by the camera should be meta-taggedas being “skiing/snowboarding” images. In yet another example, thecamera may automatically adjust the white balance setting to 7000K basedon an indication by a user.

According to one or more embodiments, a rule may indicate how one ormore images of a subject should be captured. For example, the camera maystore a rule that all images of Alice should be taken from the leftside, since Alice has a birthmark on her right arm that she prefers tohave hidden in images of her. Based on the rule, the camera may preventthe capturing of an image of Alice's right side and/or may prompt theuser to verify that he wishes to take a picture of Alice's right side.

According to some embodiments, a rule may prevent the camera fromperforming one or more operations, such as using a flash while the useris capturing images of a sporting event.

The exemplary response database shown in FIGS. 12A and 12B shows a fewexamples of how a camera may adjust a setting based on a user's responseto a question. For example, the camera adjusted its settings to“Fluorescent Light” mode based on a user responding “Fluorescent” toquestion “QUES-123478-02.” In another exemplary adjustment, the cameraadjusted the white balance setting to “5200 K” based on a userresponding “Sunny” to question “QUES-123478-05.” In another exemplaryadjustment based on a response, the camera adjusted the image sizesetting to “1600x1200” and the image compression setting to “Fine” basedon a user responding “15” to question “QUES-123478-07.”

In accordance with various embodiments of the present invention, acamera may guide a user in operating the camera based on one or moreresponses from the user. Guiding a user may include, without limitation,one or more of: recommending that a user adjust a setting, prompting auser to adjust a setting, guiding a user in composing a shot, andoutputting a message that guides a user in operating the camera.

Recommending an adjustment to a setting may include, without limitation,one or more of: outputting an indication that an adjustment to a settingis recommended, outputting a message describing an adjustment to asetting, outputting an indication of a setting to be adjusted, andoutputting an indication of a value of a setting (e.g., a current value,a recommended value). Some different types of settings on the camera aredescribed in detail herein, as are some exemplary types of adjustmentsthat may be made to settings.

According to some embodiments, the camera may guide a user byrecommending an adjustment to at least one setting based on at least oneresponse from the user. For example, the camera may recommend that auser increase his shutter speed to at least 1/250 of a second whentaking sports pictures. Response database in FIG. 7 also shows anexample of outputting a recommendation of a setting to a user based onthe user's response to a question. Based on the user's response “Yes” to“QUES-123478-09” at 4:11 p.m. on Aug. 17, 2002, the camera advised theuser to adjust the camera's shutter speed to be faster than 1/250 of asecond.

As discussed herein, in accordance with some embodiments, the camera mayactually change a setting on the camera based on a user's response to aquestion. In contrast, recommending that a user adjust a setting mayinclude simply outputting a message describing a potential adjustment toa setting, leaving it up to the user to actually adjust the setting. Forexample, a camera may output a message to a user, “I suggest that youuse a smaller aperture to ensure that both the foreground and thebackground of your photo are in focus. An aperture of f 8 or smallerwould be good for this photo.” The user may or may not make thesuggested adjustment. In another example, the camera may output amessage to a user, “If you're taking pictures of animals in the wild,then you should probably put the camera in ‘Wildlife’ mode.”

According to some embodiments, an output device may be used to output anindication of a suggestion of a setting adjustment. For example, awarning LED in the camera's viewfinder may blink to indicate to a userthat an image the user is about to capture may be underexposed (e.g.,suggesting an adjustment should be made). Note that this recommendation(i.e., the blinking LED) may simply suggest that a user make anadjustment to a setting without indicating any specific adjustment tomake. In at least one embodiment, a camera may output a messagedescribing a setting that should not be used. For example, if thecamera's flash is currently enabled and the user indicates that he iscapturing an image of a mirror, then the camera may output a message tothe user, “It is not advisable to use a flash when capturing an image ofa mirror. The flash could reflect off the mirror back at the camera,causing the image to be over exposed.”

According to some embodiments, a camera may prompt a user to adjust asetting based on at least one response from the user. Prompting a userto adjust a question may include outputting a question to a user askinghim if he would like to adjust a setting. For example, a camera mayoutput a message to a user, “Since you're taking pictures at a skislope, you should probably turn on the camera's Auto-Neutral Densityfeature.” Note that in this example, it is left to the user to determinewhether he would like to turn on the Auto-Neutral Density feature andoperate the controls of the camera to enable the feature. In anotherexample, a user may respond to a question by indicating that he is in aroom with fluorescent lights. Based on this response, the camera mayoutput a message to the user, “Would you like to put the camera in“Fluorescent Light” mode? This mode is specifically designed for roomswith fluorescent lights and will help to ensure that your images areexposed correctly.” If the user responds “Yes” to this question, thenthe camera may be adjusted to “Fluorescent Light” mode.

A camera may assist a user in adjusting a setting (e.g., without thecamera actually performing the adjustment of the setting). For example,a camera may output a message to a user, “Since you're on a sunny beach,you should probably put me in ‘Sunny Beach’ mode. Press ‘Ok’ to put thecamera in ‘Sunny Beach’ mode.” Note that in this example, the camera hasadjusted its controls to simplify for the user the process of puttingthe camera in “Sunny Beach” mode. For instance, instead of selecting“Sunny Beach” mode (e.g., from a menu of modes on the camera), all theuser has to do is press the “OK” button on the camera's touch screen.

In another example, a camera may output a message: “You may want toadjust the white balance setting on the camera based on the color oflight emitted by the light bulb in this room. Press the ‘up’ and ‘down’buttons to adjust the white balance.” In this example, the camera hassimplified the process of adjusting the white balance on the camera byautomatically enabling the “up” and “down” buttons on the camera tocontrol the white balance.

According to some embodiments of the present invention, a camera and/ora server may guide a user in composing a shot based on at least oneresponse from the user. Various types of software and/or hardware usefulin assisting a user in composing a shot are known to those skilled inthe art, including systems described in U.S. Pat. No. 5,831,670 toSuzuki, entitled “Camera capable of issuing composition information”;U.S. Pat. No. 5,266,985 to Takagi, entitled “Camera with optimumcomposition determinator”; U.S. Pat. No. 6,094,215 to Sundahl et al.,entitled “Method of determining relative camera orientation position tocreate 3-D visual images”; and U.S. Pat. No. 5,687,408 to Park, entitled“Camera and method for displaying picture composition point guides.”

According to some embodiments of the present invention, a camera maydetermine an optimum framing for a scene (e.g., with a subject slightlyoff center and an interesting tree in the background). Based on thisdetermined framing, the camera may provide instructions to a user on howto aim the camera to obtain this framing. For example, the camera mayoutput audio instructions to the use, such as, “Pan the camera a littlemore to the left . . . . Okay, that's good. Now zoom in a little bit . .. . Whoops, that's too much . . . . Okay, that's good. Now you're set totake the picture.”

In some embodiments of the present invention, a camera may include amechanism that allows the camera to aim itself. For example, a user ofthe camera may be asked to hold the camera steady, and then the cameramay adjust a mirror, lens, light sensor, and/or other optical device(s)so as to capture an image at a certain angle from the camera. Forexample, the camera may rotate a mirror five degrees to the left tocapture an image that is to the left of where a user is aiming thecamera.

According to one or more embodiments, the camera may be configured so asto be manipulated remotely (e.g., by a server). For instance, a servermay be able to view a representation of the camera's viewpoint over anetwork connection. The server may instruct a user to hold the camerasteady (e.g., via the camera's LCD display) (or direct the camera toprovide such an instruction), and then the server may adjust remotelythe camera's mirror to obtain an optimal framing of a picture.

The camera may provide directions to one or more subjects of a image.For example, a user may be capturing an image of a group of friends at arestaurant. Based on the user's response to a question and/or based onimage recognition (e.g., performed by the camera and/or a server), thecamera may provide directions relating to the group, such as, withoutlimitation: “Everybody needs to get closer together”; “Tell Alice totake a step back”; and “Bobby is giving rabbit ears to Alice.”Similarly, a camera may output directions addressed to the group ratherthan the user (e.g., using an audio speaker).

In another example of providing composition assistance to a user, acamera's viewfinder may display a blinking arrow pointing to the left toindicate to a user that he should pan left to capture the best possibleimage of a particular scene. In still another example, a user mayindicate that he is capturing an image through a glass window and wouldlike to use a flash. Based on this, the camera may provide instructionsto the user on how to compose the shot so as to avoid having the flashreflect off of the glass window.

As discussed variously herein, one or more questions may be determinedbased on a user's response to a previous question. For example, thecamera may ask a user a first question: “Are you indoors or outdoors?”If the user responds “Indoors” to this question, then the camera maystore this response and ask the user a second question based on theresponse: “What kind of lightbulbs does this room have?”

Some additional examples of determining a second question based on afirst question are provided below. Each exemplary scenario describes atleast one question output, a response by a user, and a subsequentquestion determined (e.g., by a camera, by a server) based on theresponse to at least one previous question:

-   -   (i) First question output by camera: “Are we at the beach?”        -   User's response: “Yes”        -   Second question determined: “Are you taking pictures of the            water?”    -   (ii) First question output by camera: “Are you taking a picture        of a birthday cake or candles?”        -   User's response: “Yes”        -   Second question determined: “How many candles are there on            the cake?”    -   (iii) First question output by camera: “Who are you taking a        picture of?”        -   User's response: “Alice, Bob, and Carl”        -   Second question determined: “Is Bob currently wearing the            same shirt as he was in this picture?<display picture of            Bob>”    -   (iv) First question output by camera: “Are we at the beach?”        -   User's response: “No.”        -   Second question output by camera: “Are we at a ski resort?”        -   User's response: “No”        -   Third question determined: “Are you trying to capture the            silhouette of an object?”

According to some embodiments, an entire series of questions may beoutput based on a user's response to a question. For example, inresponse to a user indicating that he is indoors, the camera may ask theuser a number of questions about the lights of the room in order todetermine what kind of lights there are, where the lights are located,and what sort of lighting effect the user is hoping to achieve. Theuser's responses to these questions may then be used to determine one ormore settings for the camera, as discussed herein.

In one example, a computing device may use a decision tree to determineone or more questions to ask a user. For example, a camera may ask auser a first question. If the user gives a first response to the firstquestion (e.g., “Yes”), then the camera may ask the user a secondquestion (e.g., the question from the “Yes” branch of the decisiontree). If the user gives a second response to the first question (e.g.,“No”), then the camera may ask the user a third question (e.g., thequestion from the “No” branch of the decision tree). This process mayrepeat until the camera determines enough information to perform one ormore actions (e.g., adjust a setting, guide a user in operating thecamera).

As discussed variously herein, a user's response to a question may be afactor in determining a question to ask the user. For instance, adetermination condition may be satisfied based on a user's response toone or more questions. For example, one determination condition may be:(location=“outdoors”) AND (image_recognition (beach_template)). Theinformation that a user is outdoors may have been received, for example,from a user as a response to a question. In another example, adetermination condition is: (location=“zoo”) AND(subject_of_image=“animal”). The information that a user is at the zoomay have been determined based on a response to a first question, andthe information that the user is capturing an image of an animal may bedetermined based on a response to a second question.

Note that the determination condition database shown in FIG. 10 includesa number of examples of the camera that describe determining a questionbased on a user's response to a previous question. In one example, ifthe camera is indoors (e.g., determined by asking “QUES-123478-01”) andthe flash is turned off, then the camera may determine to outputquestion “QUES-123478-02” (i.e., “What kind of lightbulbs does this roomhave?”) to a user. In another example, if a user answered “Alice” to“QUES-123478-03,” then the camera may output “QUES-123478-04” (i.e.,“Please use the cursor to point to Alice in this picture.”).

Some embodiments of the present invention may be advantageous in that byasking a user a plurality of questions (e.g., a series of relatedquestions), a computing device (e.g., of a camera, a server) maydetermine enough information about a scene to perform one or more otheractions (e.g., adjusting a setting, meta-tagging an image, guiding auser in operating the camera).

As described herein, a camera, server or other computing device may useone or more templates to perform image recognition on captured images.For example, the camera may store a “beach template” that may be used todetermine whether an image includes (and thus may have been captured on)a beach. As discussed herein, a wide variety of other templates arepossible.

The camera may use information provided by a user (e.g., a user'sresponse to a question) to determine a template. The template may thenbe stored and used for processing images and/or for asking questions inthe future.

For example, a camera may output a question, “Who is the subject of thisimage?” and a user may respond: “Alice.” Based on the user's responseand the image, the camera may create a template suitable for recognizingimages of Alice (e.g., an “Alice template”). At a later time, the cameramay use the “Alice template” to determine that Alice is in an image. Aquestion may then be asked based on this determination (e.g., “Who isstanding next to Alice in this picture?”).

In another example, a camera may display a plurality of images to a userand ask the user, “Were all of these images captured in a gymnasium?” Ifthe user responds “Yes” to the question, then the camera may create a“gymnasium template” based on similarities among the plurality of images(e.g., the color of the fluorescent lighting, the color of the woodfloor, etc.). If the user later returns to the gymnasium to capture moreimages, the camera may recognize that it is in the gymnasium and ask theuser a question based on this (e.g., “You're in a gymnasium, aren'tyou?”).

In another example related to a template, a camera or server may store a“group photo template” that may be used for recognizing images of groupsof people and for adjusting the settings of the camera so as to bestcapture images of groups. However, some group photos may not match thegroup photo template. For example, an image of a group of people inwhich people are lying down may not match the group photo template.Based on this, the camera may output a question to a user “Is this agroup photo?” If the user responds “Yes,” then the camera may determinea new group photo template and use this new group photo template toreplace the old group photo template. Optionally, two templates may beretained (e.g., one being for group photos where the subjects are lyingdown). In the future, the camera may recognize images of people lyingdown as group photos as well and ask appropriate questions based onthese photos.

It will be readily understood that a template may also be determinedbased on a variety of other factors (i.e., factors other than a user'sresponse to a question). According to some embodiments, a template maybe determined based on at least one image. For example, the camera maycapture a plurality of images at a ski resort and determine a “skiresort template” based on these images (e.g., based on similaritiesbetween the images). This “ski resort template” may be used to recognizeimages in which people are shown skiing or snowboarding on snow. Notethat snow provides a bright white background for such images, which maybe helpful in distinguishing images of people at a ski resort, forexample, from images of people engaged in other activities.

Some embodiments provide for determining a template based on otherindications by a user. For example, a user may use buttons on the backof the camera to select a plurality of images that are stored in thecamera and may indicate that the camera should determine a templatebased on these images. For instance, the user may select a plurality ofimages captured at a dance party and ask the camera to create a “danceparty template” based on the selected images.

As discussed herein, in accordance with various embodiments of thepresent invention a camera or other imaging device may transfer one ormore images to a second device (e.g., a server). According to someembodiments, a camera may determine whether to transmit one or moreimages to a second electronic device. For example, the camera maydetermine whether it is running low on memory and therefore should freeup some memory by transmitting one or more images to a second electronicdevice and then deleting them. Such a determination may be based on avariety of factors, including, without limitation:

-   -   (i) an amount of available memory (e.g., an amount of memory on        the camera that is free, an amount of memory on the second        device that is free)    -   (ii) an amount of bandwidth (e.g., an amount of bandwidth        available for transmitting images to the second device)    -   (iii) factors relating to capturing images    -   (iv) a user's preferences    -   (v) factors relating to images stored in memory

Similarly, according to some embodiments a camera may determine whichimages to transmit to a second device. For example, the camera may freeup some memory by transmitting images of Alice to a second electronicdevice, but keep all images of Bob stored in the camera's secondarymemory for viewing using the camera. Such a determination may be basedon a variety of factors, including, without limitation:

-   -   (i) the quality of at least one image (e.g., as measured by a        rating)    -   (ii) the compressibility of at least one image    -   (iii) image content (e.g., the subject of an image)    -   (iv) a user's preferences    -   (v) meta-data associated with at least one image (e.g., time,        location, subject, associated images)    -   (vi) an amount of available memory (e.g., an amount of memory on        the camera that is free, an amount of memory on the second        electronic device that is free)    -   (vii) an amount of bandwidth (e.g., an amount of bandwidth        available for transmitting images to the second electronic        device)

Because the bandwidth of a connection between the camera and a seconddevice may be limited, the camera may compress one or more images whentransmitting them to a second device. In addition, the camera maydetermine whether to compress an image when transmitting it to a seconddevice. For example, low quality images may be compressed before beingtransmitted to a second device, whereas high quality images may betransmitted at full resolution to the second device. Similarly, thecamera may determine how much to compress one or more images whentransmitting the one or more images to a second device. Determiningwhether to compress an image (and/or how much to compress the image) maybe based on a variety of factors, including, without limitation:

-   -   (i) an amount of available memory (e.g., an amount of memory on        the camera that is free, an amount of memory on the second        electronic device that is free)    -   (ii) an amount of bandwidth (e.g., an amount of bandwidth        available for transmitting images to the second electronic        device)    -   (iii) the quality of at least one image (e.g., as measured by a        rating)    -   (iv) the compressibility of at least one image    -   (v) image content (e.g., the subject of an image)    -   (vi) a user's preferences (e.g., indications by a user)    -   (vii) meta-data associated with at least one image (e.g., time,        location, subject, associated images)    -   (viii) other images

Note that in some embodiments a camera may delete or compress an imageafter transmitting it to a second electronic device, thereby savingmemory. Since a copy of the image may be stored on the second device(e.g., in a server database), there may be no danger of losing ordegrading the image by deleting or compressing it on the camera. Ofcourse, in some circumstances it may not be desirable to delete orcompress an image after transmitting the image to a second device. Forexample, a camera may transmit an image to a second electronic device inorder to create a backup copy of the image.

Capturing an image manually may include receiving an indication from auser that an image should be captured. Some examples of receiving anindication from a user include, without limitation: a user pressing ashutter button on a camera, thereby manually capturing an image; a usersetting a self-timer on a camera, thereby indicating that the camerashould capture an image in fifteen seconds; a user holding down theshutter button on a camera, indicating that the camera should capture aseries of images (e.g., when taking pictures of a sporting event); auser putting a camera in a burst mode, in which the camera captures asequence of three images each time the user presses the shutter button;and a user putting a camera into an auto-bracketing mode, in which thecamera captures a series of images using different exposure settingseach time the user presses the shutter button on the camera.

As discussed herein, a camera or other imaging device may capture animage automatically and then determine a question to ask a user based onthat image. For example, the image database in FIG. 8 depicts an exampleof an image “BEACHTRIP-04” that was captured automatically by a camera.It will be readily understood that capturing an image manually mayinvolve receiving an indication from a user that an image should becaptured (e.g., by the user pressing a shutter button). In contrast,automatically capturing an image may not involve receiving any suchindication. For example, the camera may capture an image automaticallywithout the user ever pressing the shutter button on the camera. Incontrast to capturing an image manually, therefore, capturing an imageautomatically may include, without limitation, one or more of thefollowing:

-   -   (i) capturing an image without a user pressing the shutter        button on the camera    -   (ii) capturing an image without an indication from a user    -   (iii) capturing an image without a direct indication from the        user    -   (iv) capturing an image without receiving an input from the user    -   (v) capturing an image without receiving an indication that the        user would like to capture the image    -   (vi) capturing an image without the user's knowledge    -   (vii) not providing an indication to a user that an image has        been captured    -   (viii) capturing an image without accessing information provided        by the user    -   (ix) capturing an image based on a condition    -   (x) capturing an image based on a condition that was not set by        a user    -   (xi) capturing an image while the camera is being held by a user    -   (xii) capturing an image independently of the user pressing the        shutter button on the camera    -   (xiii) determining whether to capture an image    -   (xiv) determining whether to capture an image automatically

According to some embodiments of the present invention, a user may ormay not be aware that an image has been captured automatically. Forexample, the user's camera may not beep, display an image that has beencaptured, or provide any other indication that it has captured an image,as is typically done by digital cameras that capture images manually.Automatically capturing an image quietly and inconspicuously may help toprevent the camera from distracting a user who is in the midst ofcomposing a shot. For example, a user may find it annoying ordistracting to have the camera automatically flash or beep when he isabout to capture an important image. In a second example, capturingimages without a user's knowledge may allow the camera to give the usera pleasant surprise at the end of the day when the user reviews hiscaptured images and finds that the camera captured sixty-eight imagesautomatically in addition to the nineteen images that he capturedmanually. In another example, a user may manually capture a plurality ofimages at a birthday party, but miss capturing an image of the birthdayboy opening one of the gifts. Fortunately, the camera may haveautomatically captured one or more images of this special event withoutthe user's knowledge.

In accordance with at least one embodiment of the present invention, acamera may capture an image automatically while a user is composing ashot. For example, a user may aim the camera at a subject and begin tocompose a shot (e.g., adjusting the zoom on the camera, etc.). While theuser is still composing the shot (i.e., before the user presses theshutter button on the camera to capture an image), the camera maycapture one or more images automatically. For example, the camera maycapture images of scenes that the user aims the camera at, even if theuser does not press the shutter button on the camera.

In some embodiments, one or more images may be captured based on acondition. For convenience, such a condition may be referred to hereinas a capture condition. Capture conditions may be useful in triggeringor enabling a variety of different functions, including, withoutlimitation: determining when to capture an image, determining what imageto capture, and determining how to capture an image.

Conditions and the performance of one or more actions based on acondition are discussed variously herein. Accordingly, it will beunderstood that capturing an image based on a condition may include,without limitation, capturing an image when a condition occurs, inresponse to a condition, when a condition is true, etc. Also, it will bereadily understood in light of discussions herein with respect toconditions, that a capture condition may comprise a Boolean expressionand/or may be based on one or more factors. Various examples of factorsupon which a condition may be based are discussed herein.

In one example of how an image may be captured automatically based on acapture condition, whenever the shutter button on the camera isdepressed halfway (i.e., a capture condition), a camera mayautomatically capture an image and store this image in RAM for furtherprocessing. In another example, a camera may include an orientationsensor that determines when the camera is being aimed horizontally andhas not moved in the last two seconds. Based on this determination, thecamera may capture an image, since a user of the camera may be composinga shot and the captured image may be useful in determining a question toask the user about the shot. According to another example, a camera mayinclude a microphone. If this microphone senses an increase in the noiselevel, then this may be a sign that an event is occurring. Based on theincrease in noise level, the camera may capture an image, which may beuseful in determining the situation and asking the user a question.

Note that automatically capturing one or more images based on a capturecondition may be particularly helpful for the camera in determining oneor more questions to ask a user. For example, whenever a user raises thecamera to a horizontal position and holds it steady, the camera maycapture an image. This image may then be used to determine anappropriate question to ask the user (e.g., a question relating to theimage that the user is about to capture). Various exemplary ways ofdetermining a question based on an image that has been captured arediscussed herein, and may be applied in accordance with some embodimentswith respect to images captured automatically.

An image that is captured based on a capture condition may be stored inmemory temporarily or permanently. For example, a camera mayautomatically delete an image that is captured automatically after thecamera has determined and output a question based on this image. Forinstance, the camera may automatically capture one or more images whilea user is composing a shot. These images may be stored in memorytemporarily and used for determining one or more questions to ask theuser. These questions may be output to the user while he is composingthe shot. The user's responses to these questions may then be used toadjust one or more settings on the camera, as discussed herein. Finally,the user may finish composing the shot and capture an image (e.g., basedon the adjusted settings). Afterwards, the automatically captured imagesmay be deleted from memory to free up space.

Alternatively, an imaging device may capture images semi-continuously(e.g., like a video camera), and a capture condition may be used toselect an image for further processing. For example, a camera maycontinuously capture images and store them temporarily in memory (e.g.,in a circular buffer of thirty images). Then, based on a capturecondition (e.g., a user pressing the shutter button halfway), the cameramay select one of the previously captured images and determine aquestion to ask the user based on this image.

It will be readily understood that an image that is capturedautomatically may be meta-tagged. For example, an image that is capturedautomatically may be meta-tagged to indicate that it can later bedeleted (e.g., if the camera starts to run out of memory).

Referring to FIG. 19, a flowchart illustrates a process 1900 that isconsistent with one or more embodiments of the present invention. Theprocess 1900 is a method for automatically capturing an image based on acapture condition. For illustrative purposes only, the process 1900 isdescribed as being performed by a camera 130. Of course, the process1900 may be performed by any type of imaging device 210.

In step 1905, the camera 130 automatically captures an image based on acapture condition. In step 1910, the camera 130 determines a questionbased on the image. In step 1915, the camera outputs the question basedon an output condition. Various types of output conditions are discussedherein. In step 1920, the camera 130 receives a response to thequestion, and in step 1925, the camera 130 adjusts a setting based onthe response.

As described variously herein, a user may provide information byresponding to a question that is output by a camera. This informationprovided by the user, as well as other information (e.g., informationfrom sensors, information from analyzing images), may be used by thecamera to determine one or more actions to perform (e.g., adjustingsettings on the camera, guiding a user in operating the camera).

According to some embodiments of the invention, information (e.g.,acquired by a camera) may expire. For example, a user may respond to aquestion by indicating that he is “at the beach.” This response may bestored in the response database (e.g., such as the one shown in FIG. 7)and an action may be performed based on the response (e.g., the cameramay be adjusted to “Sunny Beach” mode). However, at some point theinformation that the user is “at the beach” will no longer be valid. Forexample, the user may go to a restaurant to eat lunch, or go home aftervisiting the beach all day. In another example, information about theweather outside being sunny may expire at the end of the day when thesun goes down. In response to information expiring, the camera mayperform an appropriate action such as adjusting a setting on the cameraor outputting an additional question to a user. In accordance with atleast one embodiment of the present invention, a server or othercomputing device may determine that information has expired. In someembodiments, one or more actions may be performed based on theexpiration of information.

Information may expire for a variety of different reasons. Theinformation may no longer be correct, for example. For instance,information that the user is at the zoo is no longer correct if the userhas left the zoo. In another example, information may no longer beapplicable. For instance, information about how to crop an image of agroup of people may no longer be applicable if a user is not capturingan image of a group of people. In some cases, more recent informationmay be available. For example, two hours ago the weather outside mayhave been cloudy and raining, but now the weather is sunny. In someinstances, information may be time-sensitive and/or may be updatedperiodically, according to a schedule, from time to time, or at anytime.

According to at least one embodiment, information should be verifiedbefore being used again. For example, a user may indicate that he isinterested in capturing an image with a slow shutter speed and maximumdepth of field. However, these settings may or may not be appropriatefor a new scene that the user is capturing. Thus, if it is determinedthat a new scene is to be captured, for example, it may be appropriateto verify whether the user is still interested in using the samesettings (i.e., determine whether that information has expired, is stillvalid, and/or is still applicable).

In order to account for the possibility that information provided by auser has a limited lifespan, a computing device may determine wheninformation provided by a user expires and/or perform an action based onthe information's expiration. For example, the camera may ask the useranother question and/or may adjust a setting based on the expiration ofthe information.

Different pieces of information may expire at different times (e.g.,independently or each other). For example, information about thesubject(s) of one or more images that a user is currently capturing(e.g., the identities of people in a group photo) may expire when theuser ceases to aim the camera at the group of people. In anotherexample, information about the location of the camera may expire whenthe camera is moved more than one hundred feet from its originallocation. Information about the current weather conditions may expireafter two hours, for example. In some embodiments, information about auser of the camera may expire when the camera is turned off.

Alternatively, or in addition, different pieces of information mayexpire at the same time. For example, all information about capturingimages of Alice may expire if a user is now capturing an image of Bob.In another example, all information about a scene that a user wascapturing may expire if the user turns the camera off for more thanthirty minutes. In another example, all information about a user of thecamera may expire if a user presses the “Reset User Information” buttonon the camera.

A computing device may determine when one or more pieces of informationexpire based on a condition. This condition may also be referred to asan expiration condition to differentiate it from other types ofconditions described elsewhere in this disclosure. Conditions arediscussed in further detail variously herein. One skilled in the artwill understand in light of the present disclosure that some conditionsfor expiring information may be similar to conditions for determining aquestion. For example, it will be readily understood that a conditionmay be a Boolean expression and/or may be based on one or more factors.Any of the factors described herein (e.g., images, indications by auser, movement of the camera, time-related factors, state of the camera,information from sensors, characteristics of a user, information from adatabase) may also be useful in determining when information expires.Some additional examples of factors are provided below.

According to one example of determining that information has expiredbased on a condition, if a user turns the camera off (i.e., acondition), then any information about a scene that the user wascapturing may be deemed to be expired. The next time the user turns thecamera on, for example, the camera may ignore the expired informationor, alternatively, ask a user a question to verify that the expiredinformation is still relevant. In another example, if more than twohours have passed since a user provided a response to a question (i.e.,a condition), then the camera may determine that the response hasexpired and perform an action (e.g., ask the question again). In anotherexample, if a user has moved more than one thousand feet from hisoriginal location (i.e., a condition), then the camera may determinethat information relating to his original location is no longerapplicable

Information may expire (or not expire) based on one or more indicationsby a user. For example, a user may respond to a question by indicatingthat information about a scene is or is not expired. For instance, thecamera may ask a user, “Are we still at the beach?” In some exemplaryembodiments, information that affects a setting on the camera may expirebased on a user adjusting the setting on the camera. For example,information about the lighting in a room may cause the camera to adjustits white balance setting. If the user later adjusts (e.g., using acontrol) the white balance setting on the camera to “Sunny,” then thismay indicate that the user is no longer indoors and that the informationabout the lighting of the room is no longer relevant.

In another example, a user may press a “Reset Scene Information” buttonon the camera to indicate that information relating to a past scene isexpired (e.g., meaning that the camera should disregard the informationrelating to the past scene). In still another example, a user may usethe voice command “Same Scene” to indicate that information about aprevious scene has not expired (e.g., even if the camera would otherwisehave considered the information to be expired).

According to some embodiments, information related to an image mayexpire when or after an image is captured (e.g., the information aboutthe scene may only be applicable to that image). In at least oneembodiment, information about a current scene that the user is capturingmay expire when the camera is turned off.

As mentioned above, some information may expire (or not expire) based onone or more images. For example, a computing device may use a facerecognition program to analyze an image and to determine that an imageis an image of Bob. Based on this, the camera may determine, forexample, that information about capturing images of Alice is expired. Inanother example, a user may have indicated that he is at the beach.However, thirty minutes later, the camera may determine that one or moreimages captured recently do not match any of the “beach templates”stored by the camera. Based on this, the camera may determine that theinformation that the user is on the beach may have expired.

Some types of information may expire (or not) based on the state of thecamera. For example, a camera may keep track of how many images havebeen captured since a piece of information was received. After athreshold number of images (e.g., ten images) have been captured, theinformation may expire. In another example, information may expirewhenever the camera is turned off. Note that the camera may be turnedoff based on an indication by a user (e.g., the user presses the powerbutton on the camera) and/or based on other conditions (e.g., the cameramay automatically turn itself off after five minutes of inactivity).

According to some embodiments, information may expire when the camera'sbatteries are replaced, when the camera is plugged into a wall outlet torecharge, or when images are downloaded from the camera (e.g., forstorage on a personal computer).

Information may expire based on a user. For example, a camera may storeinformation about its current user (e.g., the user's identity, theuser's preferences and habits when capturing images, a list of imagesthat have already been captured by the user). If the camera is latergiven to a new user, information about the previous user may expire,since it is not applicable to the new user. For example, Alice and Bobmay share a camera. When Alice is capturing images using the camera, thecamera may adjust one or more settings based on Alice's userpreferences. If Alice then hands the camera to Bob, the informationabout Alice's user preferences may expire and be replaced withinformation about Bob's user preferences.

According to some embodiments of the present invention, information mayexpire or not expire based on one or more of a variety of time-relatedfactors. Some examples of time-related factors are described hereinwithout limitation. Information may expire, for example, after aduration of time. For instance, information that a user provides about ascene may expire after thirty minutes unless it is reaffirmed by theuser (e.g., by indicating the information is still valid, by providingadditional information about the scene). In another example, informationmay expire at a specific time. For instance, information about whetherthe sky is sunny, partially cloudy, or overcast may expire at 6:34 p.m.(e.g., when the sun goes down). According to at least one embodiment,information may expire based on a condition existing for a duration oftime. For example, information about the lighting in a room may expireif the camera's light meter reads bright (outdoors) light for more thanthirty seconds.

Examples of information expiring or not expiring based on informationfrom sensors include, without limitation: determining a location,determining an orientation of a camera, and determining informationabout light. For example, a camera may use GPS device to determine howfar it has been moved from a location where a user provided a responseto a question. If the camera has been moved more than a thresholddistance from the location where the user provided the response, thenthe information provided by the response may be determined to beexpired. In another example, a camera may use an orientation sensor todetermine when a user is aiming the camera at a scene. If the user stopsaiming the camera at the scene for longer than ten seconds, forinstance, then the camera may determine that the user is no longercapturing images of the scene and determine that information about thescene is expired. According to some embodiments, an imaging device mayuse a light sensor to determine the color of light that is shining onthe camera. If the color of light shining on the camera is 5200K(daylight), then the camera may determine that information indicatingthe camera is under fluorescent light bulbs (4000K) is expired.

According to one or more embodiments of the present invention,information expiring or not expiring may be based on one or morecharacteristics of a user include. For example, a user may be in thehabit of turning his camera off anytime he does not anticipate capturingan image in the next minute. Based on this, the camera may adjust itsconditions for expiring information so that information about a scenedoes not expire unless the user turns the camera off for an extendedperiod of time (e.g., fifteen minutes). In another example, a user mayprefer that he not be asked the same question twice in close succession(e.g., within ten minutes). Based on this, the camera may prolong thetime that it takes for a piece of information to expire (e.g., to morethan ten minutes). In this way, the camera may effectively postponeasking the user a second question relating to the information inaccordance with the user's preference.

In some embodiments of the present invention, information may expire ornot expire based on information from one or more databases. In oneexample, a first piece of information may be based on the validity ofone or more second pieces of information. If the second pieces ofinformation expire, then the first piece of information may also expire.For example, a camera may store two pieces of information: a) the camerais currently indoors, and b) the room has fluorescent lighting. If thefirst piece of information (i.e., the camera is indoors) expires becausethe user moves outdoors, then this may cause the second piece ofinformation (i.e., that the room has fluorescent lighting) to expirealso, since it is unlikely that there is also fluorescent lightingoutdoors. However, it will be recognized that the inverse may not betrue. That is, the expiration of the information that the room hasfluorescent lighting may not mean that the information that the camerais indoors has also expired. For example, a user of the camera may havejust moved to another room.

In another example, an imaging device (e.g., a PDA with an integratedcamera) may determine that information has expired based on a change toan image template stored in a database. For example, the imaging devicemay determine a revised image template for Alice (e.g., a revised “Alicetemplate”) because Alice has put on a blue sweater over her green tanktop. Based on this, the imaging device may determine that informationabout the subject of the image (e.g., a girl in a green tank top) isexpired.

In yet another example, a camera may store information about when thesun comes up or goes down. When the sun goes down, the camera may thenexpire any information about the current weather conditions (e.g.,sunny, cloudy, etc.). In another example, a camera may receive weatherreports via a radio link (and optionally may store an indication of thereceived information). For instance, the camera may receive an updatedweather report indicating that the weather outside is no longer sunnyand is now raining. Based on this updated report, the camera maydetermine that information indicating that light is shining on subjectssitting outside should be expired.

When a piece of information expires (e.g., as determined using anexpiration condition), an imaging device and/or computing device mayperform one or more of a variety of different actions, including,without limitation: ceasing to perform an action (e.g., an action thatwas performed based on the information), outputting a question,adjusting a setting, meta-tagging an image, guiding a user in operatingthe camera, storing information, and any combination thereof.

If a piece of information expires, then this may mean that an actionthat a camera was performing based on the information is no longerappropriate. Therefore, in accordance with some embodiments, a cameramay cease to perform the action (and may optionally perform a secondaction instead). Examples of ceasing to perform an action based onexpiring information include, without limitation: readjusting a setting,ceasing to meta-tag images, and ceasing to guide a user in operating acamera.

As discussed variously herein, an imaging device such as a camera mayadjust a setting (e.g., the mode of the camera) based on a user'sresponse to a question. According to various embodiments of the presentinvention when the user's response to the question expires (e.g., basedon an expiration condition), the camera may adjust the setting again(e.g., returning the setting to its original value or adjusting to a newvalue). In a more detailed example, the camera may adjust itself to “SkiSlope” mode based on a user's indication that he is on a ski slope. Whenthe ski slope closes (e.g., 4:00 p.m.) the camera may determine that theuser's indication that he is on a ski slope has expired. Accordingly,the camera may readjust itself to cancel “Ski Slope” mode and put thecamera in “Default” mode instead.

According to some embodiments, a camera may cease to meta-tag imageswhen information expires. In some embodiments, a camera may cease tometa-tag images with information that has expired. For example, thecamera may meta-tag one or more images based on a user's response to aquestion, as described herein. If the information upon which themeta-data is based expires, then the camera may cease to associate thatmeta-data with images. For example, the camera may receive informationfrom a user that the user is capturing images of a group of people:Alice, Bob, Carly, and Dave. This information may be used to meta-tagthe captured images. However, when the user puts his camera down fortwenty seconds, the information about the group of people may expire(e.g., based on an expiration condition). Accordingly, future imagescaptured by the camera will not be tagged as images of Alice, Bob,Carly, and Dave.

Expiration of information may cause an imaging device to cease guiding auser in operating the camera. For example, a camera may guide a user inoperating the camera based on the user's response to a question, asdescribed herein. For instance, the camera may guide a user in adjustingthe shutter speed of the camera based on a user's indication that he iscapturing images of wildlife. If an image recognition program running onthe camera (or on a server in communication with the camera, asdiscussed herein) later determines that the user is about to capture animage of a person, then the camera may cease to provide instructions tothe user about how to capture images of wildlife (e.g., because thewildlife-related information has effectively expired).

According to at least one embodiment of the present invention, a cameramay output a question to a user based on information expiring. Forexample, in response to a piece of information expiring, the camera mayask a user a question relating to the information that expired. Theuser's response to this question may be helpful in replacing theinformation that expired and/or in guiding the camera in performingadditional actions to assist the user. The following, withoutlimitation, are some exemplary scenarios related to outputting aquestion to a user based on information expiring:

-   -   (i) A determination condition may be based on information        expiring. For example, the camera may determine to output the        question, “Are you indoors or outdoors?” based on the        determination condition: expired (indoors_or_outdoors_response).    -   (ii) Information about a scene may expire when a user stops        aiming a camera at the scene. The camera may then remain idle,        for example, until the user begins to aim the camera at a new        scene, at which point the camera may determine that a) the        information about the old scene has expired and b) the camera        does not have any information about the new scene. Based on        this, the camera may determine and output an appropriate        question to the user.    -   (iii) Information about the lighting of a scene (e.g., daylight,        tungsten light bulb, fluorescent light bulb) may expire whenever        a light sensor on the camera determines that the light color or        intensity of a scene has changed dramatically. Accordingly, the        camera may output the question, “What type of lighting does this        scene have?” whenever the light sensor causes information to        expire.    -   (iv) Information about whether a scene includes running water        may expire (e.g., based on image processing). Based on this        expiration, the camera may output a question to the user, “Does        this image include water?”        Other types of situations in which it may be desirable to output        a question based on information expiring may be readily apparent        to those skilled in the art in light of the present disclosure.

As discussed herein, according to some embodiments of the presentinvention, a camera may adjust one or more settings based on informationexpiring. Some examples, without limitation, are provided below, andvarious other exemplary ways in which a setting may be adjusted arediscussed herein.

In one example, information about the current lighting conditions mayexpire. Based on this, the camera may adjust its settings toauto-exposure and auto-white balance. In another example, informationabout who is the current user of the camera may expire. Based on this,the camera may revert to its default user preferences. In anotherexample, information about what object in the field of view (e.g., theforeground, the background) a user would like to focus on may expire.Based on this expiration, the camera may adjust its focus settings(e.g., to five-region, center-weighted auto-focus).

In yet another example, information about a user being on a boat mayexpire. Based on this, the camera may adjust its digital stabilizationsetting to “regular.” Information about the weather outside being sunnymay expire because the current time of day is after sunset, for example.Based on this, the camera may assume that it is indoors or nighttime andturn its flash on.

Various exemplary ways in which a camera, for example, may meta-tag animage are discussed herein. According to at least one embodiment of thepresent invention, a device such as a camera or server may meta-tag oneor more images based on information expiring. For example, informationabout the subject of an image may expire. Based on this, a camera maymeta-tag an image as “Subject: Unknown.” According to some embodiments,a user can later review the image and provide meta-data about thesubject of the image. In another example, information about a locationof the camera may expire. After determining the location information hasexpired, the camera may omit location information when meta-tagging animage (or may not meta-tag an image at all).

Various exemplary ways in which a camera, for example, may guide a userin operating the camera are discussed herein. In accordance with one ormore embodiments of the present invention, a camera may guide a user inoperating the camera based on information expiring. For instance,information about whether the subject of an image is in the shade or thesun may expire. Based on this, the camera output a message to guide auser: “If your subject is in the shade, you may want to adjust the whitebalance setting on the camera to 7000K or use a flash. If your subjectis in the sun, you may want to adjust the white balance setting on thecamera to 5200K and make sure that your subject is facing towards thesun.” In another example, information about the subject of one or moreimages may expire. In response, the camera may output a message to auser: “You can meta-tag your images with information about theirsubjects by pressing the ‘Meta-Tag’ button and saying the name of thesubject.”

The camera may store information based on information expiring. That is,according to some embodiments of the present invention, the camera maystore a first piece of information based on the expiration of a secondpiece of information. For example, information about the camera beingunderwater may expire based on a conductivity sensor on the body of thecamera. Based on information received via the conductivity sensor, thecamera may store an indication that it is not underwater.

If an expiration condition occurs, then related information may bedetermined to be expired (and optionally the camera may perform one ormore actions), as described herein. As also discussed herein,information may expire based on other information expiring. For example,if a camera is turned off for more than sixty minutes, then theinformation that the camera is outdoors may expire. Based on this, theinformation that the weather is sunny and that the user is on a beachmay also expire. A single condition might cause multiple pieces ofinformation to expire. For example, the information that an imageincludes a body of water and the piece of information that the userprefers to have no reflections may both expire if an image does notmatch a water template or if a user presses the “Reset ImagePreferences” button on the camera.

Different actions may be performed based on what causes information toexpire. For example, if information indicating that the subject of animage is Alice expires because a user is no longer aiming the camera orbecause thirty seconds have elapsed, then the camera may stopmeta-tagging images as “Subject: Alice.” However, if that informationexpires because an image recognition program (e.g., executed by thecamera, executed by a server in communication with the camera) does notrecognize the subject of an image as being Alice, then the camera maystop meta-tagging the image and ask: “Who are you photographing?” Thus,an action (e.g., determining and/or outputting a question) may beperformed based on information expiring and/or based on the particularcircumstance(s) that caused the information to expire.

As discussed herein, a camera or other device (e.g., a server) may storean expiring information database that stores information aboutconditions that may cause information to expire. One example of anexpiring information database is shown in FIG. 14. Note that the exampleof the expiring information database shown in FIG. 14 may store at leastone expiration condition for each piece of information that is stored bythe camera.

Referring to FIG. 20, a flowchart illustrates a process 2000 that isconsistent with one or more embodiments of the present invention. Theprocess 2000 is a method for performing an action based on informationexpiring. For illustrative purposes only, the process 2000 is describedas being performed by a camera 130. Of course, the process 2000 may beperformed by any type of imaging device 210 and/or computing device 220.

In step 2005, the camera 130 receives information related to use of thecamera 130. For example, the camera determines or otherwise receives(e.g., from a sensor, from a server 110, from a user) any of the varioustypes of information described herein. For example, the camera 130receives an indication that it is raining, a user preference, a signalthat the memory is low, etc. In step 2010, the camera 130 determines anexpiration condition for the information. For example, the camera 130determines that the piece of information should expire after thirtyminutes.

In step 2015, the camera 130 stores an indication of the information andan indication of the expiration condition (e.g., in an expirationcondition database). In step 2020, the camera 130 determines if theinformation has expired (e.g., based on the expiration condition). Ifthe information has not expired, the camera 130 performs a first actionin step 2025. If the information has expired, the camera 130 performs asecond action based on the information expiring (e.g., a correspondingaction indicated in an expiration condition database).

In accordance with various embodiments of the present invention, acamera may output a question to a party other than a user of the camera.For example, a camera may output a question to a human subject (i.e., aperson) of one or more images captured (or to be captured) by thecamera.

Questions may be output to subjects of images for a variety of differentpurposes. For example, a question may be output to verify that an imagewas captured properly. For instance, a camera may be used to capture animage of a group of people (e.g., Alice, Bob, and Carly). Immediatelyafter capturing the image of the group of people, the camera may outputa question to the group: “Did anybody blink?” If one or more people inthe group answer “Yes” to the question, then the camera may capture oneor more additional images of the group, in the hope of capturing atleast one image of the group in which nobody is blinking.

According to another example, a preference of a subject may bedetermined. For example, the camera may output a question to a subjectof an image: “Do you want this to be a close-up shot from the waist up,or a full-body shot that includes your feet?” The camera may adjust oneor more settings (e.g., a zoom setting) based on the subject's responseto this question. Performing one or more actions based on a subject'spreferences (e.g., adjusting a setting, meta-tagging an image) may beparticularly appropriate for subjects who have strong feelings about howimages of them should be captured.

In some embodiments, an imaging device may assist a subject in posing.For example, a camera may output a question to a subject of an image:“It looks like there's a piece of paper sticking to your shoe. Do youwant to remove this before the photo is taken?” Based on the user'sresponse, the camera may then pause to allow the subject to remove thepiece of paper from his foot.

It will be readily understood that a question may be output to a subjectusing an output device. Many types of output devices are discussedherein, and others will be readily apparent to those skilled in the artin light of the present disclosure. Some examples include, withoutlimitation, an audio speaker, a video monitor, and a wireless handhelddevice.

For example, a camera may include an audio speaker to play an audiorecording of a question loud enough for a subject of an image to hearthe question. In another example, the camera may use a HyperSonic Sound®directional sound system by American Technology Corporation to output aquestion to a subject that may not be heard by other subjects of theimage, the user, or bystanders. In another example, a camera may includean LCD display or other type of video monitor that faces (or may beconfigurable to face) a subject of the camera (e.g., away from a user ofthe camera). The camera may use this LCD, for instance, to display atext question to the subject.

In yet another example of an output device, a subject of a camera maycarry a wireless handheld device (e.g., a remote control, a cell phone,a PDA) that communicates with the camera (e.g., using a infrared orradio communications link). The camera may output a question to thesubject by transmitting the question to the wireless handheld device.The wireless handheld device may then display the question to thesubject (e.g., using an audio speaker, LCD display screen, or otheroutput means). Other embodiments operable for outputting a question to asubject of an image may be similar to those described herein foroutputting a question to a user of the camera.

According to some embodiments, a subject may respond to a question usingan input device, such as, without limitation, a microphone, an imagesensor, or a wireless handheld device. For example, a subject of animage may respond to a question by speaking the answer aloud. The cameramay use a microphone and voice recognition software to record anddetermine the user's response to the question. In another example, asubject of an image may respond to a question by making an appropriatehand signal to the camera (e.g., thumbs-up for “Yes,” thumbs-down for“No”). The camera may use an image sensor to capture one or more imagesof the subject making the hand signal and then process the images usingan image recognition program to determine the subject's response to thequestion.

In another example, a subject of an image may respond to a questionusing a wireless handheld device (e.g., a remote control, a cell phone,a PDA) operable to communicate with a camera. For example, a subject ofan image may press a button on his PDA or speak into a microphone on hiscell phone to provide a response to a question. The PDA or cell phonemay then transmit an indication of the response to the camera (e.g., viaa communication network).

A variety of exemplary actions that may be performed based on a user'sresponse to a question are discussed herein (e.g., adjusting a setting,meta-tagging an image, outputting a second question). Other actions arealso possible. Additional actions that may be performed by the camerabased on a user's response to a question include automatically capturingan image and managing images stored in memory.

As discussed herein, an imaging device may be configured to capture animage automatically. According to some embodiments, a camera mayautomatically capture one or more images based on a user's response to aquestion. For example, if the camera asks a user, “Are we at a footballgame?” and the user responds, “Yes,” then the camera may automaticallycapture one or more images whenever the players on the football fieldare moving. Various exemplary processes for automatically capturing animage (e.g., based on a condition) are discussed herein.

Automatically capturing one or more images based on a user's response toa question may comprise one or more of, without limitation: determiningwhether the camera should automatically capture one or more images basedon a user's response to a question; determining what images the camerashould automatically capture based on a user's response to a question;and determining how the camera should treat the one or moreautomatically-captured images (e.g., compressing them) based on a user'sresponse to a question.

One way for a user's response to a question to affect a process ofautomatically capturing images is for the camera to adjust a settingrelating to automatically capturing images. One example of a setting onthe camera that may relate to automatically capturing images is acondition for automatically capturing images. For example, the cameramay automatically capture an image when a condition is true. A user'sresponse to a question may be a factor that affects a condition. Inanother example, a threshold value for determining whether to store animage may relate to the automatic capturing of images. For example, thecamera may capture an image and then determine a rating for the imagebased on the quality of the image. If the rating of the image is higherthan a threshold value, then the camera may store theautomatically-captured image. If the rating is worse than the thresholdvalue, then the automatically-captured image may be compressed ordeleted. In another example, a parameter that affects how much anautomatically-captured image is compressed may be adjusted. For example,the camera may automatically compress an automatically-captured imagebased on a compression setting. For instance, images with greatercompression settings may be compressed more and images will lessercompression settings may be compressed less.

According to one or more embodiments, a camera may manage one or moreimages stored in memory based on a user's response to a question.Managing images stored in memory may include, without limitation, one ormore of:

-   -   (i) Uploading an image using a network. For example, a camera        may include a radio modem, cell phone, or other wireless network        connection that allows the camera to transmit images to a second        electronic device (e.g., a computer server, a laptop computer, a        cell phone). Uploading an image using a network may be        particularly useful in sharing images with other people (e.g.,        friends of a user) or freeing up memory on the camera (e.g.,        since an image may not be stored on the second electronic        device).    -   (ii) Modifying one or more images. For example, a camera may use        image editing software to modify one or more images that are        stored in memory. Examples of modifications that may be made to        images include cropping, removing red-eye, color balancing,        removing shadows, removing objects from the foreground or        background, adding or removing meta-data, and combining images        into a panorama.    -   (iii) Compressing or deleting one or more images. For example,        as discussed herein, a camera may automatically compress or        delete one or more images stored in memory in order to make room        for additional images that may be captured by the camera.

The following, without limitation, are some exemplary scenarios relatedto managing images stored in memory based on a user's response. Eachscenario includes a question (e.g., asked by a camera), a response froma user, and an action:

-   -   (i) Question: “How many more images are you planning on        capturing?”        -   Response from a user: “20-30”        -   Action: Use a wireless network connection (e.g., 3G cellular            network) to upload images from the camera to a central            computer. Then delete the images that have been uploaded,            thereby freeing up space in the camera's memory for the            20-30 additional images that the user plans on capturing.    -   (ii) Question: “Are you capturing an image of a group of        people?”        -   Response from a user: “Yes”        -   Action: Process the captured image to remove shadows that            fall across people's faces and red-eye that may have            resulted from using a flash.    -   (iii) Question: “Which images are more important to you: a)        images of Alice, or b) images of Bob?”        -   Response from user: “a) images of Alice”        -   Action: Sort the images in memory into images of Alice and            images of Bob. Re-compress all the images of Alice using a            JPEG compression setting of 80%, thereby reducing the file            sizes of these images and freeing up memory space in the            camera. Do not perform any additional compression on the            images of Bob that are stored in memory.

One or more embodiments of the present invention may enable a camera orother imaging device to determine more easily a scene to be captured inan image and a user's intentions for capturing an image. Suchdeterminations will enable some types of users to more easily adjust thesettings on their cameras, making capturing images a simpler and moreenjoyable process. In addition, some embodiments of the invention mayallow a user to capture better images, even if he does not have adetailed knowledge of photography.

Although the methods and apparatus of the present invention have beendescribed herein in terms of particular embodiments, those skilled inthe art will recognize that various embodiments of the present inventionmay be practiced with modification and alteration without departing fromthe teachings disclosed herein.

1. A camera comprising: means for recording an image; means fortransmitting the image to a remote computer for determining at least onequestion based on the image; means for receiving from the remotecomputer an indication of at least one question; and means foroutputting to a user a representation of the at least one question.2-10. (canceled)